DAS-2411: Write Inherited Coordinates so that panoply can plot all variables in output file #23
Conversation
The xarray to_NetCDF() function appears to enforce alignment of dimensions in a hierarchical tree, including writing those dimensions into groups below that inherited them. by including the keyword write_inherited_coords=True on the function call the dimension variables are also included. This allows panoply to find the needed values to plot a variable
flamingbear
requested review from
joeyschultz,
lyonthefrog,
vutrannasa and
a team
| @@ -115,7 +115,7 @@ def amend_in_file_metadata( | |||
| # whole `xarray.DataTree` in one operation. Making this write variables | |||
| # and group separately reduces the memory usage, but makes the | |||
| # operation slower. (See Harmony SMAP L2 Gridder implementation) | |||
| datatree.to_netcdf(output_file_name) | |||
| datatree.to_netcdf(output_file_name, write_inherited_coords=True) | |||
There was a problem hiding this comment.
Is there a JIRA to investigation/fix a better solution?
There was a problem hiding this comment.
Nope. I'm going to do some quick investigation today if I get a chance.
There was a problem hiding this comment.
Actually this is going to end up involved. But I've discovered why HRS is failing after this update and will have to implement configuration changes there also. before generating the regression output.
|
Test: PASS Test Steps:
branch)
|
|
I'm going to close this. This fix was working around pydata/xarray#10241 before I could discover if that was a bug or design decision. Since this is fixed in xarray v25.8.0. I'm going to add h5netcdf to the deps and remove it when we can add the next version of xarray that includes the fix to pydata/xarray#10654. with the PR pydata/xarray#10656 |
Description
A short description of the changes in this PR.
Write dimension variables into inherited groups. This is a band-aid to xarray's roundtrip of opening a datatree with inherited dimensions getting added to the child groups. What results is a dimension on the group that doesn't exist. Because there's a keyword parameter to the
datatree.to_netcdffunction this may be a design choice. I am going to look and see. But for now, writing the dimension vars into the child groups allows panoply to open the files properly.This change will require regression hashes to be updated as the output files will be different for many requests.
Jira Issue ID
DAS-2411
Local Test Steps
Pull this branch. Build and verify all tests pass. Deploy the image to your local Harmony-In-A-Box.
Run this curl command against uat and localhost to generate the file against the current implementation (uat) and the updated (localhost)
and
Open the files and verify that the localhost variables are plotable in panoply but the uat's are not.
PR Acceptance Checklist
CHANGELOG.mdupdated to include high level summary of PR changes.docker/service_version.txtupdated if publishing a release.