-
Notifications
You must be signed in to change notification settings - Fork 202
add capability to dynamically generate GSI *info files #3472
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jswhit2
wants to merge
110
commits into
NOAA-EMC:develop
Choose a base branch
from
jswhit2:build_gsinfo
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+384
−278
Open
Changes from 21 commits
Commits
Show all changes
110 commits
Select commit
Hold shift + click to select a range
3182e17
introduce new env var USE_BUILD_GSINFO
a97e131
add scripts for generating GSI info files when USE_BUILD_GSINFO=YES
e4094ad
update
87cae24
Merge branch 'develop' into build_gsinfo
16182d3
Merge branch 'develop' into build_gsinfo
3fdd95c
use OBS_INPUT envar to point to obs_input namelist block table text file
a8a3473
fix for bad hirs coeff files (workaround until new crtm)
b589b9b
add OBS_INPUT and HIRS_FIX
1ab1ef6
Merge branch 'NOAA-EMC:develop' into build_gsinfo
49b5ed8
move scripts from scripts to ush, add shebang
dd77bb4
address shell linting errors
0e87789
address shell linting errors
3fcc4f1
address shell linting errors
1e7b9f8
fixed crtm2 hirs2 coeff files now included in build_gsinfo
339dfdb
address shell linting errors
25ab1b0
address more shell linting errors
25b7871
scripts now in USHgfs
11ad3f0
add check for existence of satellites file
1dd3450
update
901be87
Merge branch 'develop' into build_gsinfo
ac22e7d
update
62607ab
Merge branch 'develop' into build_gsinfo
e572cd4
make executable, use BUILD_GSINFO_DIR, remove tabs
9fbe816
use BUILD_GSINFO_DIR
1e53c68
create_*info.sh now executable.
e252934
use OBS_INPUT_DIR
2814d4f
fix typo
b27e1d9
add ignore_2mQM
0c3167e
Merge branch 'develop' into build_gsinfo
c45fb22
Merge branch 'develop' into build_gsinfo
aerorahul 7e25699
Update scripts/exglobal_diag.sh
aerorahul 1700e86
Update ush/create_satinfo.sh
aerorahul 124ea8d
Update ush/create_ozinfo.sh
aerorahul 90ea718
Update ush/create_ozinfo.sh
aerorahul 3232b11
Update ush/create_satinfo.sh
aerorahul 88df316
Add double quotes to variables in exglobal_atmos_analysis.sh
DavidHuber-NOAA 72aac5c
Add double quotes to variables in exglobal_diag.sh
DavidHuber-NOAA 96cbc66
Convert unneeded cat to < operator
DavidHuber-NOAA a7b1b22
update scripts as suggested in PR, have create_convinfo turn on 2m ob…
e42e82e
Merge branch 'develop' into build_gsinfo
jswhit a34a23c
use ncep ozone by default
a3db7fa
Address shellcheck issues.
DavidHuber-NOAA 518e891
More shellcheck issues.
DavidHuber-NOAA 8571872
Merge remote-tracking branch 'emc/develop' into upd/gsinfo
DavidHuber-NOAA 57ca8ed
Address shellcheck issues throughout the touched scripts
DavidHuber-NOAA 23fd055
update
adc0c69
Make a generic script to handle *stat files
DavidHuber-NOAA 18933ca
Make create_gsi_info.sh executable
DavidHuber-NOAA a5d769a
Remove now-unused create_*info.sh files
DavidHuber-NOAA 860e194
Merge branch 'develop' into build_gsinfo
DavidHuber-NOAA f98b090
Merge branch 'build_gsinfo' into upd/gsinfo
DavidHuber-NOAA 5b33ba0
Update dev/parm/config/gfs/config.anal
DavidHuber-NOAA 277b3a7
Update ush/create_gsi_info.sh
DavidHuber-NOAA 09d41be
Update scripts/exglobal_atmos_analysis.sh
DavidHuber-NOAA 3ef0a37
Update ush/create_gsi_info.sh
DavidHuber-NOAA b69786a
Convert grep to egrep for easier-to-understand searching
DavidHuber-NOAA 1bd07d9
Use grep -E instead
DavidHuber-NOAA df0ac5d
Add ozone header; fix regex
DavidHuber-NOAA 9085f45
Group the header echos
DavidHuber-NOAA e6260c5
Update scripts/exglobal_atmos_analysis.sh
DavidHuber-NOAA fdccb58
Be more explicit about where we're working
DavidHuber-NOAA d8949ab
Update ush/create_gsi_info.sh
DavidHuber-NOAA 31db8bb
Set 183 obs to monitor
DavidHuber-NOAA 183eb72
Tie BUILD_2M_OBS to DO_GSISOILDA
DavidHuber-NOAA 5f4233e
Also monitor 183 q
DavidHuber-NOAA 7cd535e
Apply patch from Jeff
DavidHuber-NOAA eaefd75
Merge pull request #1 from DavidHuber-NOAA/upd/gsinfo
jswhit 6f2d23a
remove outdated comments
dca61d6
Remove 'set' from create script (already in preamble)
DavidHuber-NOAA fae6cd3
Add the directory to the create call
DavidHuber-NOAA 4f2a8df
Fix ozone info file creation; write satellite function
DavidHuber-NOAA 3ba3d2c
update
54e1a59
Add override option for BUILD_2M_OBS
DavidHuber-NOAA 422a1ba
Update build_gsinfo
DavidHuber-NOAA d0fe259
Fix build directory variable name
DavidHuber-NOAA 8167654
Add parm links for gsinfo files
DavidHuber-NOAA ee61b30
Point to parm rather than fix for build_gsinfo
DavidHuber-NOAA 4fc6f73
Add gsinfo to ignore list
DavidHuber-NOAA 11d6566
Add missing }
DavidHuber-NOAA b0b1fea
Merge remote-tracking branch 'origin/build_gsinfo' into update/gsi_stat
DavidHuber-NOAA 00d5a8f
Merge branch 'NOAA-EMC:develop' into build_gsinfo
jswhit 9ca7754
implement reviewer suggestions
a3c92a9
set BUILD_2M_OBS
3affd14
Update ush/create_gsi_info.sh
jswhit 495729e
Update ush/create_gsi_info.sh
jswhit c20e045
Update ush/create_gsi_info.sh
jswhit 40bfb6c
Merge branch 'develop' into build_gsinfo
jswhit 48598f4
Link in the obs_input directory as well
DavidHuber-NOAA 6eca255
Add hirs_fix to parm from build_gsinfo
DavidHuber-NOAA d9f6a0b
Update build_gsinfo
DavidHuber-NOAA a115e59
Update build_gsinfo
DavidHuber-NOAA cc09a13
Merge remote-tracking branch 'emc/develop' into update/gsi_stat
DavidHuber-NOAA ae22aab
Merge in Jeff's updates
DavidHuber-NOAA 23aa8b0
Fix merge oops
DavidHuber-NOAA bc562a0
Correct the usage prints
DavidHuber-NOAA 4aa903c
change BUILD_2M_OBS to USE_2M_OBS
c9fecd0
Merge branch 'develop' into build_gsinfo
jswhit 64dffbb
Merge branch 'build_gsinfo' into update/gsi_stat
94d92b5
Merge pull request #2 from DavidHuber-NOAA/update/gsi_stat
399af6b
Merge branch 'develop' into build_gsinfo
DavidHuber-NOAA 30c7e51
Merge branch 'develop' into build_gsinfo
jswhit 4438e6b
update
b1b8be0
Merge branch 'develop' into build_gsinfo
jswhit2 d8a0471
move * within quotes
c3518bb
fix quoting that was causing analdiag to fail with tcp obs
11e44c7
Update build_gsinfo
DavidHuber-NOAA 370d6d4
Merge branch 'build_gsinfo' of github.com:jswhit2/global-workflow int…
DavidHuber-NOAA 0b7f01f
Merge branch 'develop' into build_gsinfo
jswhit 59f0cdb
Update gsi hash
DavidHuber-NOAA 39f4fd0
Add me to gitmodules
DavidHuber-NOAA File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
#! /usr/bin/env bash | ||
# create global_convinfo file for a given date | ||
#date=$1 | ||
cd "${FIXgfs}/gsi/build_gsinfo/convinfo" || exit | ||
# currently using a single convinfo | ||
cat merged_convinfo.txt | ||
#usedate="" | ||
#for datex in [1-2]*; do | ||
# if [[ ${date} -ge ${datex} ]]; then | ||
# usedate=$datex | ||
# fi | ||
#done | ||
## cat matching date file, or quit if date not found | ||
#if [[ ${usedate} != "" ]]; then | ||
# cat ${usedate} | ||
#else | ||
# echo "date not found" | ||
# exit 1 | ||
#fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
#! /usr/bin/env bash | ||
# create global_ozinfo file for a given date | ||
date=$1 | ||
aerorahul marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
# header | ||
echo '! For mls data, pressure and obs errors are pulled from bufr, so not listed here' | ||
echo '! sens/instr/sat lev use pressure gross obs b_oz pg_oz' | ||
echo '! error error variational qc' | ||
# loop over satellites | ||
cd "${FIXgfs}/gsi/build_gsinfo/ozinfo" || exit 1 | ||
grep -v '^ *#' satellites | while IFS= read -r sat | ||
do | ||
# find matching date | ||
usedate="" | ||
for f in "${sat}"/*; do | ||
if [[ ${f} != "${sat}/readme" ]]; then # skip readme file | ||
datex=$(basename "${f}") | ||
if [[ ${date} -ge ${datex} ]]; then | ||
usedate=${datex} | ||
fi | ||
fi | ||
done | ||
# cat matching date file, or quit if date not found | ||
if [[ ${usedate} != "" ]]; then | ||
cat "${sat}/${usedate}" || exit 1 | ||
else | ||
echo "date not found for ${sat}" | ||
exit 1 | ||
fi | ||
done |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
#! /usr/bin/env bash | ||
# create global_satinfo file for a given date | ||
date=$1 | ||
aerorahul marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
# header | ||
echo '!sensor/instr/sat chan iuse error error_cld ermax var_b var_pg icld_det icloud iaerosol' | ||
# loop over satellites | ||
cd "${FIXgfs}/gsi/build_gsinfo/satinfo" || exit 1 | ||
grep -v '^ *#' satellites | while IFS= read -r sat | ||
do | ||
# find matching date | ||
usedate="" | ||
for f in "${sat}"/*; do | ||
|
||
if [[ ${f} != "${sat}/readme" ]]; then # skip readme file | ||
datex=$(basename "${f}") | ||
if [[ ${date} -ge ${datex} ]]; then | ||
usedate=${datex} | ||
fi | ||
fi | ||
done | ||
DavidHuber-NOAA marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
# cat matching date file, or quit if date not found | ||
if [[ ${usedate} != "" ]]; then | ||
cat "${sat}/${usedate}" || exit 1 | ||
else | ||
echo "date not found for ${sat}" | ||
exit 1 | ||
DavidHuber-NOAA marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
fi | ||
done |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.