No longer clear all encoding in set_zarr_encoding

[TomNicholas]

I found that the way set_zarr_encoding clears all other encoding fields was frustrating, so this PR changes the function to only change the fields it needs to.

I also think this function should be made explicitly public.

[maxrjones]

Thanks for the PR @TomNicholas! Do you mind sharing more information about what specifically was cleared from the encoding that was frustrating?

I can see the value of keeping other encoding fields, but also expect that clearing the encoding may have arisen from differing conventions for fill values which can cause issues with serializing to Zarr (similar to pydata/xarray#7722 but with the Zarr backend). @andersy005 or @norlandrhagen do you know the history / motivation behind fully clearing the encoding fields?

[TomNicholas]

Do you mind sharing more information about what specifically was cleared from the encoding that was frustrating?

Specifically the chunking mostly. I create a dataset with some chunking that was chosen for dask purposes, then am asked to save it to disk with another chunking pattern chosen for access via carbonplan-maps. If I pass the desired chunks via the encoding kwarg to ds.to_zarr it only uses that encoding, ignoring what I set in set_zarr_encoding. If I set the chunks encoding on the dataset object then call set_zarr_encoding it silently clears the chunks encoding. I suppose I could use set_zarr_encoding before altering it to add the chunks encoding in, but I would rather that set_zarr_encoding wasn't order-dependent like that.

I guess I could alternatively call ds.chunk first (which is what seems to happen in ndpyramid.utils.set_zarr_encoding_and_metadata).

[maxrjones]

Do you mind sharing more information about what specifically was cleared from the encoding that was frustrating?

Specifically the chunking mostly. I create a dataset with some chunking that was chosen for dask purposes, then am asked to save it to disk with another chunking pattern chosen for access via carbonplan-maps. If I pass the desired chunks via the encoding kwarg to ds.to_zarr it only uses that encoding, ignoring what I set in set_zarr_encoding. If I set the chunks encoding on the dataset object then call set_zarr_encoding it silently clears the chunks encoding. I suppose I could use set_zarr_encoding before altering it to add the chunks encoding in, but I would rather that set_zarr_encoding wasn't order-dependent like that.

I guess I could alternatively call ds.chunk first (which is what seems to happen in ndpyramid.utils.set_zarr_encoding_and_metadata).

This may be another docs issue. How are you setting the chunking pattern? The recommended way when using pyramid_reproject is by setting pixels_per_tile for spatial dims and other_chunks for non-spatial dims, while keeping the total chunk size ~1MB or less (https://nasa-impact.github.io/zarr-visualization-report/approaches/dynamic-client/e2e-results-zarr-version.html), such that external rechunking shouldn't be necessary.

[TomNicholas]

This may be another docs issue. How are you setting the chunking pattern? The recommended way when using pyramid_reproject is by setting pixels_per_tile for spatial dims and other_chunks for non-spatial dims

Oh - I had not seen the other_chunks option! That probably would have been fine.

I think that because I keep creating a store (an expensive operation) then later trying to modify it somehow I'm hitting issues just because I'm not using the code top-to-bottom in quite the intended way. I think my original complaint might not be valid.