Skip to content

Recommendations for Zarr readers #159

@langmore

Description

@langmore

I'd recommend a couple changes

  • Change xr.open_zarr(path) to xr.open_zarr(path, chunks=None). Since you're not using Dask, this will speed things up. Without it, datasets with millions of chunks will be super slow to slice...as Dask will try to rebuild a graph (that you'll never use). It also blows up memory. FWIW, xbeam.open_zarr already passes chunks=None.
  • The ARCO-ERA5 dataset is awesome, but they named it ....zarr-v3 (even though it's Zarr2). Your Xarray readers don't open this correctly because they try to auto-detect the filename ending. There are multiple ways around this.

BTW, I would just send a PR, but my previous PRs often sit around unsubmitted...so I figure it's easier for you to do it internally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions