Skip to content

Add Python code for AMIP ancillary generation#141

Open
penguian wants to merge 3 commits into
mainfrom
121-amip
Open

Add Python code for AMIP ancillary generation#141
penguian wants to merge 3 commits into
mainfrom
121-amip

Conversation

@penguian

@penguian penguian commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Closes #121

See also https://github.com/ACCESS-NRI/CMIP7-Input/tree/scen7-amip
https://github.com/ACCESS-NRI/ancillary-file-science/tree/1-port-cmip7-amip-code-for-esm16-site-nci
https://code.metoffice.gov.uk/trac/roses-u/browser/d/q/8/1/9/cleanup?rev=360148
https://code.metoffice.gov.uk/trac/roses-u/browser/d/q/8/1/9/trunk?rev=360149

This pull request adds the cmip7_AM_amip_generate.py Python code for the AM_seaice_ancil_amip and AM_sst_ancil_amip tasks. This code takes the NetCDF files produced by the AM_ukesm_ancil and produces the final ancillary files. This code has also been incorporated into the https://github.com/ACCESS-NRI/CMIP7-Input/tree/scen7-amip branch, which includes all changes from https://github.com/ACCESS-NRI/CMIP7-Input/tree/127-n2-and-o3-for-esm16-h-and-vl

The code was tested by running https://code.metoffice.gov.uk/trac/roses-u/browser/d/q/8/1/9/cleanup?rev=360148 which uses the scne7-amp branch and produced the files

/g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.02/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.02/sst_amip_n96_gregorian.anc
/g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.02/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.02/seaice_amip_n96_gregorian.anc

The run logs are in /scratch/tm70/pcl851/cylc-run/u-dq819.cleanup/run10/log/job/1/AM_*_ancil_amip/

@penguian penguian linked an issue Jul 3, 2026 that may be closed by this pull request
@penguian penguian self-assigned this Jul 3, 2026
@penguian

penguian commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator Author

After the latest commit, I ran https://code.metoffice.gov.uk/trac/roses-u/browser/d/q/8/1/9/trunk?rev=360149 with all ancillaries other than AMIP set to false:

[pcl851@gadi-login-04 u-dq819.trunk]$ svn diff
Index: rose-suite.conf
===================================================================
--- rose-suite.conf     (revision 360149)
+++ rose-suite.conf     (working copy)
@@ -2,27 +2,27 @@
 SITE = 'nci-gadi'
 ISO_DATE_TODAY = '$(isodatetime -f CCYY.MM.DD)'
 
-ANCIL_CREATE_AEROSOL = true
+ANCIL_CREATE_AEROSOL = false
 ANCIL_CREATE_AMIP = true
-ANCIL_CREATE_CO2 = true
-ANCIL_CREATE_GHG = true
-ANCIL_CREATE_NITROGEN = true
-ANCIL_CREATE_OZONE = true
-ANCIL_CREATE_SOLAR = true
-ANCIL_CREATE_VOLCANIC = true
+ANCIL_CREATE_CO2 = false
+ANCIL_CREATE_GHG = false
+ANCIL_CREATE_NITROGEN = false
+ANCIL_CREATE_OZONE = false
+ANCIL_CREATE_SOLAR = false
+ANCIL_CREATE_VOLCANIC = false
 
 EXPS = ['AM', 'EH', 'HI', 'PI', 'PM', 'SM']
 MAIN_EXPS = ['HI', 'PI', 'SM']
 USE_EXP = {'AM': True,
-           'EH': True,
-           'HI': True,
-           'PI': True,
-           'PM': True,
-           'SM': True,}
+           'EH': False,
+           'HI': False,
+           'PI': False,
+           'PM': False,
+           'SM': False,}
 
 SCENARIOS = ['h', 'vl']
-USE_SCEN = {'h': True,
-            'vl': True,}
+USE_SCEN = {'h': False,
+            'vl': False,}
 
 TESTING = false

This produced the files

g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.03/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.03/sst_amip_n96_gregorian.anc
/g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.03/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.03/seaice_amip_n96_gregorian.anc

The logs are in /scratch/tm70/pcl851/cylc-run/u-dq819.trunk/run94/log/job/1/*AM*/01:

[pcl851@gadi-login-04 u-dq819.trunk]$ more /scratch/tm70/pcl851/cylc-run/u-dq819.trunk/run94/log/job/1/*AM*/01/job.err
::::::::::::::
/scratch/tm70/pcl851/cylc-run/u-dq819.trunk/run94/log/job/1/AM_seaice_ancil_amip/01/job.err
::::::::::::::
Using the cylc session localhost

Loading cylc/8.6.3
  Loading requirement: mosrs-setup/2.0.1
/home/851/pcl851/cylc-run/u-dq819.trunk/run94/lib/python/esm1p6_ancil/cmip7_ancil_ukesm.py:24: FutureWarning: Ignoring a datum in netCDF load for consistency with existing behaviour. In a future version of Iris, this datum will be applie
d. To apply the datum when loading, use the iris.FUTURE.datum_support flag.
  return iris.load_cube(filepath)
::::::::::::::
/scratch/tm70/pcl851/cylc-run/u-dq819.trunk/run94/log/job/1/AM_sst_ancil_amip/01/job.err
::::::::::::::
Using the cylc session localhost

Loading cylc/8.6.3
  Loading requirement: mosrs-setup/2.0.1
/home/851/pcl851/cylc-run/u-dq819.trunk/run94/lib/python/esm1p6_ancil/cmip7_ancil_ukesm.py:24: FutureWarning: Ignoring a datum in netCDF load for consistency with existing behaviour. In a future version of Iris, this datum will be applie
d. To apply the datum when loading, use the iris.FUTURE.datum_support flag.
  return iris.load_cube(filepath)

@penguian penguian requested a review from Copilot July 3, 2026 06:10
@penguian penguian changed the title 121 amip Add Python code for AMIP ancillary generation Jul 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new AMIP ancillary-generation entrypoint for ESM1.6 that reads UKESM-produced NetCDF forcings, applies common coordinate fixes, and writes final UM ancillary outputs into the CMIP7 target directory structure.

Changes:

  • Added cmip7_AM_amip_generate.py to generate AMIP SST/sea-ice ancillaries from UKESM-produced NetCDF inputs.
  • Introduced cmip7_ancil_ukesm.py helper module for parsing/loading/fixing UKESM ancillary NetCDF cubes.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
CMIP7/esm1p6/atmosphere/cmip7_ancil_ukesm.py New shared helper for UKESM NetCDF ancillary ingestion and coordinate adjustments.
CMIP7/esm1p6/atmosphere/amip/cmip7_AM_amip_generate.py New AMIP generator wiring together parsers, UKESM load/fix, and ancillary save pathing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +45 to +46
gregorian=False,
replace_bounds=True,

# Load the CMIP7 datasets
ukesm_cube = load_cmip7_ukesm(args)
# Match the ESM1.5 mask
Comment on lines +8 to +13
def ukesm_parser():
parser = ArgumentParser(add_help=False)
parser.add_argument("--ukesm-ancil-dirpath")
parser.add_argument("--ukesm-netcdf-filename")
parser.add_argument("--save-filename")
return parser

@penguian penguian Jul 3, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. I left the ozone/cmip7_ozone.py code in deliberately, partly because it does not affect AMIP generation, and partly because it might make for a good first refactoring task for Valentina.

@penguian

penguian commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator Author

The checksums have been stable since the run on 2026.06.26.

[pcl851@gadi-login-04 u-dq819.trunk]$ for var in seaice sst; do for f in /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.*/modern/amip/atmosphere/boundary_conditions/global.N96/2026.*/${var}_amip_n96_gregorian.anc;do md5sum $f;done; done
e96784e440971724c45d508a3b7cef9d  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.04.21/modern/amip/atmosphere/boundary_conditions/global.N96/2026.04.21/seaice_amip_n96_gregorian.anc
096eca3ee8f50685bc1c393036a5907a  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.26/modern/amip/atmosphere/boundary_conditions/global.N96/2026.06.26/seaice_amip_n96_gregorian.anc
096eca3ee8f50685bc1c393036a5907a  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.29/modern/amip/atmosphere/boundary_conditions/global.N96/2026.06.29/seaice_amip_n96_gregorian.anc
096eca3ee8f50685bc1c393036a5907a  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.01/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.01/seaice_amip_n96_gregorian.anc
096eca3ee8f50685bc1c393036a5907a  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.02/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.02/seaice_amip_n96_gregorian.anc
096eca3ee8f50685bc1c393036a5907a  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.03/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.03/seaice_amip_n96_gregorian.anc
0a396a15a78800f78b5108dce5e0c1d2  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.04.21/modern/amip/atmosphere/boundary_conditions/global.N96/2026.04.21/sst_amip_n96_gregorian.anc
0d234603969c6eead17f586b0b354736  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.26/modern/amip/atmosphere/boundary_conditions/global.N96/2026.06.26/sst_amip_n96_gregorian.anc
0d234603969c6eead17f586b0b354736  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.29/modern/amip/atmosphere/boundary_conditions/global.N96/2026.06.29/sst_amip_n96_gregorian.anc
0d234603969c6eead17f586b0b354736  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.01/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.01/sst_amip_n96_gregorian.anc
0d234603969c6eead17f586b0b354736  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.02/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.02/sst_amip_n96_gregorian.anc
0d234603969c6eead17f586b0b354736  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.03/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.03/sst_amip_n96_gregorian.anc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Produce AMIP SST and sea-ice ancillaries for ESM1.6

2 participants