netCDF SpcCoeff: load NLTECoeff/ACCoeff sibling files, add guard test, bump to v3.1.5 - #364
Merged
Merged
Conversation
… prefer canonical coeff dirs The substructure-file lookup at the end of SpcCoeff_netCDF_ReadFile was declared CHARACTER(LEN(Filename)) :: sub_filename. The .SpcCoeff. -> .NLTECoeff. substitution is one character longer, so the trailing 'c' of '.nc' was silently truncated and File_Exists returned false. The NLTE sibling load quietly no-oped for every absolute-path caller, leaving SpcCoeff%NC default-initialized and producing several-K brightness-temperature drift on NLTE-active channels for all CrIS/AIRS/IASI sensors. ACCoeff was unaffected because 'ACCoeff' is shorter than 'SpcCoeff'. Buffer is now LEN(Filename)+16, with comments explaining why. Also teach the loader to honor the canonical fix-tree layout: when the SpcCoeff path contains /SpcCoeff/netCDF/, substitute that segment with /NLTECoeff/netCDF/ or /ACCoeff/netCDF/ to find the sibling. Falls back to same-directory sibling lookup for non-canonical (flat) layouts. Verified end-to-end: cris-fsr_n21, cris-fsr_n20, mhs_n19, and a flat temp-dir layout all load with the expected substructure associations. (cherry picked from commit 98c6815)
Unit + forward-model ctest for the NLTECoeff/ACCoeff sibling load in the netCDF SpcCoeff reader. Read-level: iasi616_metop-b must pick up its co-located NLTECoeff.nc (27 channels) and amsua_n19 its ACCoeff.nc, and both loads must still succeed, with the substructures left empty, when the siblings are absent. Forward-level: with vs without the NLTECoeff sibling, daytime BTs must differ by >0.5 K on NLTE channels only (measured 4.17 K at solar zenith 45) and night-time BTs must be identical, confirming the correction is applied and solar-driven. Stages testinput_no_siblings/ alongside testinput/ with the same coefficient set minus the sibling files. The forward assertions fail against a reader that skips the siblings, so this test guards the sibling-load fix against regression.
v3.1.5: the netCDF SpcCoeff reader now loads the NLTECoeff/ACCoeff sibling files, previously skipped silently, which disabled the non-LTE and antenna corrections for netCDF-coefficient users.
BenjaminTJohnson
added a commit
that referenced
this pull request
Aug 25, 2026
Brings in v3.1.5 (PR #364: netCDF SpcCoeff sibling-load fix + test_NLTE_Verification) and the CMake download/untar refactors (PR #354, #356). Conflict resolution: - CMakeLists.txt, VERSION.cmake, README.md: keep the 3.2.0 version strings; add the v3.1.5 line to the README release history. - SpcCoeff_netCDF_IO.f90: keep this branch's version. develop's fix is a subset of it (same sibling load, minus the "no sibling found" INFORMATION messages). - test/CMakeLists.txt: keep the fix_REL-3.2.0.0 tarball/checksum on top of develop's combined download+untar block; keep the netCDF-only coefficient staging list and add the iasi616_metop-b SpcCoeff/NLTECoeff/TauCoeff set the new test needs; port the testinput_no_siblings staging block to the netCDF filenames since the 3.2.0 tarball ships no .bin coefficients.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
All 3.1.x releases silently skip the
<sensor>.NLTECoeff.nc/<sensor>.ACCoeff.ncsibling files when reading netCDF SpcCoeff, so the non-LTE correction and the MW antenna correction are disabled with no warning on the netCDF-coefficient path (the binary reader streams both inline and is unaffected). The observable signature is a ~2 K daytime OmF spike in the 4.3 um CO2 band for hyperspectral IR sounders — confirmed end-to-end for IASI (GSI OmF stats, standalone forward runs, and a JEDI/UFO H(x) A/B on real obs: night dH(x) = 0 exactly, day up to 4.5 K, NLTE channels only).Changes
feature/btj_REL-3.2.0: after reading the netCDF SpcCoeff, look for the siblings in the canonicalfix/ACCoeff/netCDF//fix/NLTECoeff/netCDF/directories first, then co-located next to the SpcCoeff file; sensor-consistency checks on load; absence is not an error (graceful degradation). The conflict resolution also brings the twoACCoeff_netCDF_IO/NLTECoeff_netCDF_IOimports the module needs (added by an earlier commit on the feature branch).test_NLTE_Verification: read-level assertions (iasi616_metop-b NLTECoeff, 27 channels; amsua_n19 ACCoeff; both with-sibling and without-sibling) plus a forward-model A/B — daytime BTs must shift >0.5 K on NLTE channels only (measured 4.17 K at SZA 45) and night-time BTs must be identical. Stagestestinput_no_siblings/next totestinput/. The test fails against the pre-fix reader, so the sibling load is guarded against regression.Verification
Fresh Release build of this branch: new test passes 12/12 assertions in ~2 s; full ctest suite 196/196. Negative control (siblings hidden from
testinput/) fails exactly the sibling-load assertions with exit 1.Note for forward-porting
The
fix_REL-3.2.0.0tarball used by the REL branches has no co-located siblings and noTauCoeff/ODPS/Little_Endian/iasi616_metop-b.TauCoeff.bin; when merging this test forward, the staging lists should switch to the canonicalNLTECoeff/netCDF//ACCoeff/netCDF/paths and a netCDF TauCoeff.