Open
Conversation
…sets, and in inference
b8raoult
changed the title
aaron-hopkinson
requested changes
Mar 18, 2026
Contributor
aaron-hopkinson
left a comment
aaron-hopkinson
left a comment
There was a problem hiding this comment.
After the anemoi-transform PR has been merged, we will need a new release of that package and to update the pyproject.toml here.
Code generally looks fine - just a few typos and one missing method in anemoi-transform.
| for field in fields: | ||
| units = field.metadata("units", default=None) | ||
| if units is not None: | ||
| assert False, units |
Contributor
There was a problem hiding this comment.
Should this line be removed? (Or the one below won't execute)
| (18, "0-3/3-6/6-9/9-12/12-15/15-18"), | ||
| ] | ||
|
|
||
| case ("e6", "oper", _): |
Contributor
There was a problem hiding this comment.
Unrelated code change?
| def variables_metadata(self) -> dict[str, Any]: | ||
| """Retrieve the metadata for the variables.""" | ||
| return _fields_metatata(self.variables, self._cube) | ||
| return _fields_metatata(self.variables, self._cube, self._past_units) |
Contributor
There was a problem hiding this comment.
Suggested change
| return _fields_metatata(self.variables, self._cube, self._past_units) | |
| return _fields_metadata(self.variables, self._cube, self._past_units) |
| dict | ||
| The metadata dictionary. | ||
| """ | ||
| def _fields_metatata(variables: tuple[str, ...], cube: Any, units_seen: dict) -> dict[str, Any]: |
Contributor
There was a problem hiding this comment.
Suggested change
| def _fields_metatata(variables: tuple[str, ...], cube: Any, units_seen: dict) -> dict[str, Any]: | |
| def _fields_metadata(variables: tuple[str, ...], cube: Any, units_seen: dict) -> dict[str, Any]: |
| if isinstance(ds, pd.DataFrame): | ||
| raise ValueError( | ||
| "Did you forget meant to build a tabular dataset? Did you forget to specify 'format: tabular' in your recipe?" | ||
| "Did you forget meant to build a tabular dataset? Did you forget to specify 'layout: tabular' in your recipe?" |
Contributor
There was a problem hiding this comment.
Suggested change
| "Did you forget meant to build a tabular dataset? Did you forget to specify 'layout: tabular' in your recipe?" | |
| "Did you mean to build a tabular dataset? Did you forget to specify 'layout: tabular' in your recipe?" | |
| other[variables[i]]["process"] = process | ||
| other[variables[i]]["period"] = (startStep, endStep) | ||
|
|
||
| units = Units.to_canonical(f.metadata("units", default=None)) |
Contributor
There was a problem hiding this comment.
NB: anemoi-transform branch Unit class doesn't contain a to_canonical class method (yet?)
github-project-automation
bot
moved this from To be triaged
to Under Review
in Anemoi-dev
github-actions
bot
added
dependencies
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add units to datasets, so we can check them when combining datasets, and in inference.
concat: 💣 Variable cp has multiple units: m and kg m**-2in addition to the actual unit, accumulation periods and equivalent are added:
m;accumulation(6:00:00)meansmetersandaccumulated over 6hWhat problem does this change solve?
What issue or task does this change relate to?
Additional notes
As a contributor to the Anemoi framework, please ensure that your changes include unit tests, updates to any affected dependencies and documentation, and have been tested in a parallel setting (i.e., with multiple GPUs). As a reviewer, you are also responsible for verifying these aspects and requesting changes if they are not adequately addressed. For guidelines about those please refer to https://anemoi.readthedocs.io/en/latest/
By opening this pull request, I affirm that all authors agree to the Contributor License Agreement.