-
Notifications
You must be signed in to change notification settings - Fork 45
feat: refactor dataset create #379
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…o feat/recipe-generator
…o feat/recipe-generator
Co-authored-by: Marek Jacob <[email protected]>
Co-authored-by: Marek Jacob <[email protected]>
Co-authored-by: Matthew Chantry <[email protected]>
Co-authored-by: Matthew Chantry <[email protected]>
Co-authored-by: Matthew Chantry <[email protected]>
Co-authored-by: Matthew Chantry <[email protected]>
Co-authored-by: Matthew Chantry <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will approve once the comment on the copy command is resolved, because I think that will break something.
We need to either revert the change to the copy command or reopen PR #151 on anemoi-utils
Co-authored-by: Marek Jacob <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for your work and the consideration of my comments.
Co-authored-by: Aaron Hopkinson <[email protected]>
Co-authored-by: Aaron Hopkinson <[email protected]>
Description
This is a rewrite of the code that parses the recipe and invoke the sources and filters in the right order. These changes are merely a simplification of the old version and are 100% backward compatible.
The input family of actions has been refactored such that all the standalone action files, e.g. concat are consolidated into a single action.py.
Additional related additions.
What problem does this change solve?
This change is in preparation of adding support for observation in anemoi-dataset[create].
What issue or task does this change relate to?
Additional notes
As a contributor to the Anemoi framework, please ensure that your changes include unit tests, updates to any affected dependencies and documentation, and have been tested in a parallel setting (i.e., with multiple GPUs). As a reviewer, you are also responsible for verifying these aspects and requesting changes if they are not adequately addressed. For guidelines about those please refer to https://anemoi.readthedocs.io/en/latest/
By opening this pull request, I affirm that all authors agree to the Contributor License Agreement.