add capability to dynamically generate GSI *info files #3472
Conversation
jswhit2
requested review from
CoryMartin-NOAA and
RussTreadon-NOAA
as code owners
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
ShellCheck found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
|
@jack-woollen @jswhit2 Jeff I didn't figure out how to update your branch with the convinfo file containing complete satwnd definitions. Instead you can copy from /work2/noaa/da/jwoollen/RAEXPS/scripts/2021ozn1/build_gsinfo/convinfo/merged_convinfo.txt. |
There was a problem hiding this comment.
@jswhit2 To include the soil analysis in the reanalysis / scout runs, we'll need to make changes to the convinfo and anavinfo. For the NRT system, those changes are here.
Can you please add these changes into your [conv/anav]info files? I'm not sure if it's better to include them always, or as an option.
|
@ClaraDraper-NOAA I think you forgot to include the link. Would you be able to tell us specifically which lines to add to the. reanalysis version of convinfo (https://github.com/NOAA-PSL/build_gsinfo/blob/main/convinfo/merged_convinfo.txt)? The anavinfo changes won't be needed for the scout runs until we start running ensemble DA. |
|
@DavidHuber-NOAA curious about why you made this change. Seems like the correct place to look is |
|
@jswhit that change goes along with this one: jswhit2@8167654. The latter creates links from the The reasons for this change set is that we can link directly to the |
|
@jswhit2 @ClaraDraper-NOAA I ran a test case that generates the 2m observation info files along with the other GSI *info files for a C96 analysis and for cycle 2025090100. The run directory for the analysis can be found here: Can you take a look at this directory and verify that everything looks as it should (and/or ping others to take a look)? I would like to start the process of merging upstream submodule PRs so we can move this PR forward. The It is worth noting that the |
I don't think we can let this get merged in if it breaks |
|
@CoryMartin-NOAA just clarifying that this only breaks ‘analdiag’ if the ‘build_gsinfo’ is enabled, which it is not by default. I’m fine with continuing to work on this issue if that’s still a blocking issue. |
|
@DavidHuber-NOAA got it, I misunderstood. As long as it passes with the default/old configuration, that's fine with me |
OK that makes sense. Thanks @DavidHuber-NOAA |
Unfortunately, I don't have RDHPCS access yet (after my recent re-hire) so I can't see those files. I do still have access to orion, so if you can copy the files there (including the log from the failed analdiag step) I can take a look. |
|
@jswhit2 sure, I've copied over the expdir, comroot, and anal.2297506 directories to Orion here: |
|
the analdiag step fails because a diag_tcp file is not created. This should be fixed by jswhit2@c3518bb |
|
Checked the satinfo, ozinfo and convinfo files in @DavidHuber-NOAA's test and compared to the files in gfsv17_historical. Aside from one error in avhrr3_metop-c (which is now fixed in build_gsinfo-fix the files are functionally equivalent (but not identical). The only differences are for instruments that no longer exist, or are turned off. |
|
@jswhit thanks for the fixes! That did get the gdas_analdiag job to run and the rest of the cycle ran successfully as well. |
8 participants
Description
Currently config.anal has an if/then/else loop that sets env vars SATINFO, OZINFO, CONVINFO that contain paths to different gsi info files for different periods over the last several years. For reanalysis, we need a solution that works back to 1979. Rather than add to the if/then/else block in config.anal, I have created a set of scripts that generate *info files dynamically given a date (https://github.com/NOAA-PSL/build_gsinfo). What is needed is a way to use those scripts in global-workflow. Note this is only needed for GSI - for JEDI the needed functionality is included in observation_chronicle (https://github.com/NOAA-EMC/jcb-gdas/tree/develop/observation_chronicle/atmosphere).
Resolves #3293
Requires NOAA-EMC/GSI-fix#28
Enabled via USE_BUILD_GSINFO env var that can be set to YES in config.base (default is NO).
3 new scripts added (create_satinfo.sh, create_ozinfo.sh, create_convinfo.sh). These scripts generate the GSI *info for a given analysis date using data from build_gsinfo (which will live inside GSI-fix once NOAA-EMC/GSI-fix#28 is merged).
The OBS_INPUT table in the GSI namelist is removed from exglobal_atmos_analysis.sh to allow for separate options for NCEP ops and reanalysis. Both versions are included as text files in build_gsinfo/obs_input. The OBS_INPUT env var can be used to choose which version do use. The NCEP ops version is the default in config.anal.
A workaround for NOAA-EMC/GSI#752 is included in exglobal_atmos_analysis.sh (pointing to a separate directory for HIRS coefficient files). This hack can be removed once NOAA-EMC/GSI#783 is merged.
Type of change
Change characteristics
Is this a breaking change (a change in existing functionality)?NO
Does this change require a documentation update? YES
Does this change require an update to any of the following submodules? YES
How has this been tested?
Checklist