-
Couldn't load subscription status.
- Fork 167
Modified config_archive.xml to archive CAM+DART files. #1302
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kdraeder
wants to merge
5
commits into
ESCOMP:cam_development
Choose a base branch
from
kdraeder:cam_st-arch_dart
base: cam_development
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
f69921e
Added code to make st-archive handle DART files
kdraeder 1084a90
Draft of entry to satisfy development workflow recommendations
kdraeder a42315e
Added line to archive .i. files as both restarts and history
kdraeder 1f31cd7
resolve merge conflicts
jedwards4b e268bbc
still trying to fix ChangeLog
jedwards4b File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wondering about the implications of removing this line:
<hist_file_extension>i\..*\.nc$</hist_file_extension>on other CAM jobs. I will admit that I am by no means an expert on this file and archiving in general, but it appears that we'd no longer be archiving intantaneous history files? Should this line perhaps remain?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That line was replaced by a <rest_file_extension>i\..* line.
This line handles the CAM "initial file", which is the only '.i.' file that I know of.
There may be '.i[a-z0-9]+.' files (someday), but they would be handled differently.
But I would like to hear the opinion of CAM developers about this.
DART needs to have the .i. files archived with the restarts because that's the file
containing the model state which DART needs to use to start CAM at the beginning of each DA cycle.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I want to be sure we are talking about the same thing.
instantaneous files are the
.h0i.,.h1i., ... which is different from the initial file.i..Where will these files go?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe that the .h[0-9]i. files will still be handled by the xlm line:
<hist_file_extension>h\d*.*\.nc(\.gz)?$</hist_file_extension>which I parse as "h+any_digit+any_number_of_any_characters".
The '.i.' file is currently handled by the line:
<hist_file_extension>i\.\d.*\.nc(\.gz)?$</hist_file_extension>I believe that the parsing code adds '.' before the regex listed in these .xml files,
so this line only applies to '.i.' and not to the .h[0-9]i. files.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto @cecilehannay . We need to know if this change moves the location of history output and breaks everyone's analysis scripts. Fixable, yes, but it would be good to know in advance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my test the .h0i. files ended up in archive/atm/hist, archive/rest/$date, and in $rundir.
I only requested 1 CAM history file.
The .i. files ended up in archive/rest/$date and $rundir.
That's what I intended, but someone else should test it too, especially if I failed to create a CAM file type.
The archive directory is /glade/derecho/scratch/raeder/St_BHISTC_LTso-SE_st-arch/archive.
Thanks for looking into this!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cecilehannay - Would you be able to checkout @kdraeder's branch and make a quick run similar to ones you are currently making and make sure all that all the history files end up where they should?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cecilehannay - I have two CAM checkouts for you to test:
Please let us know what your runs indicate for both the history and the initial condition files (and any other files for that matter if they change)