test_numpy_arrays_to_inlined_kerchunk_refs intermittently deadlocks in CI (kerchunk -> zarr sync under h5py's phil lock)

[TomNicholas]

virtualizarr/tests/test_integration.py::test_numpy_arrays_to_inlined_kerchunk_refs intermittently hangs until pytest-timeout kills it at 300s. It has hit main (523caa4, test-py312) and multiple PR runs, in different parametrizations (vars_to_inline0 and vars_to_inline1) and different jobs (test-py312-build, minimum-versions-build), so it isn't tied to one environment or parameter.

The hang is in the line that computes the expected value, before any virtualizarr code runs:

expected = SingleHdf5ToZarr(netcdf4_file, inline_threshold=int(inline_threshold)).translate()

Stack at timeout:

test_integration.py:93            .translate()
kerchunk/hdf.py:170               self._h5f.visititems(self._translator)
h5py/_hl/group.py:771             return h5o.visit(self.id, proxy)
h5py/_objects.pyx:54              with_phil.wrapper          # <- h5py global lock held
h5py/h5o.pyx:367                  cb_obj_simple
kerchunk/hdf.py:535               za = self._zroot.require_array(...)
zarr/core/group.py:2830           self._sync(self._async_group.require_array(...))
zarr/core/sync.py:152             wait([future], timeout=None)   # <- blocks forever
threading.py:355                  waiter.acquire()

kerchunk calls zarr's sync API from inside an h5py visititems callback, so it holds h5py's global phil lock while blocking indefinitely on zarr's event-loop thread (timeout=None). Whatever leaves that loop unable to complete the future, the result is an unbounded wait rather than an error.

No virtualizarr code is on the stack — this is a kerchunk/zarr/h5py interaction — but it makes our CI red at random, so it's ours to contain.

Possible directions:

• Mark the test @pytest.mark.flaky so it runs in the existing non-blocking flaky-tests job.
• Set a finite zarr.config async timeout in tests so a stuck loop fails fast with a traceback instead of burning 300s.
• Stop recomputing expected via kerchunk at test time (store a fixture), removing the kerchunk-inside-phil path entirely.
• Report upstream to kerchunk, since calling a sync-over-async API from within visititems is the underlying hazard.