Python code for aerosol, greenhouse gas, nitrogen, ozone, solar and volcanic forcings for ScenarioMIP for ACCESS-ESM1.6 by penguian · Pull Request #135 · ACCESS-NRI/CMIP7-Input

penguian · 2026-06-02T05:58:22Z

Closes #127. Closes #132. Closes #133. Closes #134.

The Python code produces ancillaries for aerosol, greenhouse gas, nitrogen, ozone, solar and volcanic forcings for ScenarioMIP for ACCESS-ESM1.6 when run with the current u-dq819/scenarios@358302 Cylc 8 workflow and the current 3-port-cmip7-ozone-code-for-esm16 branch of (https://github.com/ACCESS-NRI/ancillary-file-science
See also https://github.com/ACCESS-NRI/ancillary-file-science/issues/3

penguian · 2026-06-05T01:54:36Z

See #137 for the failing readthedocs check.

penguian · 2026-06-15T11:53:05Z

The latest run of u-dq819.scenarios completed successfully.

See gadi:/scratch/tm70/pcl851/cylc-run/u-dq819.scenarios/run77 for the run logs.

The branches created are:

The file output is:

$ tree /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.15/
/g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.15/
├── modern
│   ├── historical
│   │   ├── atmosphere
│   │   │   ├── aerosol
│   │   │   │   └── global.N96
│   │   │   │       └── 2026.06.15
│   │   │   │           ├── BC_1849_2023_cmip7.anc
│   │   │   │           ├── Bio_1849_2023_cmip7.anc
│   │   │   │           ├── OCFF_1849_2023_cmip7.anc
│   │   │   │           └── scycl_1849_2023_cmip7.anc
│   │   │   ├── forcing
│   │   │   │   └── resolution_independent
│   │   │   │       └── 2026.06.15
│   │   │   │           ├── TSI_CMIP7_ESM
│   │   │   │           └── volcts_cmip7.dat
│   │   │   └── land
│   │   │       └── biogeochemistry
│   │   │           └── global.N96
│   │   │               └── 2026.06.15
│   │   │                   └── Ndep_1849_2023_cmip7.anc
│   │   └── forcing
│   │       └── global.N96
│   │           └── 2026.06.15
│   │               └── ozone_1849_2023_cmip7.anc
│   └── pre-industrial
│       ├── atmosphere
│       │   ├── aerosol
│       │   │   └── global.N96
│       │   │       └── 2026.06.15
│       │   │           ├── BC_1850_cmip7.anc
│       │   │           ├── Bio_1850_cmip7.anc
│       │   │           ├── OCFF_1850_cmip7.anc
│       │   │           └── scycl_1850_cmip7.anc
│       │   └── land
│       │       └── biogeochemistry
│       │           └── global.N96
│       │               └── 2026.06.15
│       │                   └── Ndep_1850_cmip7.anc
│       └── forcing
│           └── global.N96
│               └── 2026.06.15
│                   └── ozone_1850_cmip7.anc
└── scenarios
    ├── h
    │   ├── atmosphere
    │   │   ├── aerosol
    │   │   │   └── global.N96
    │   │   │       └── 2026.06.15
    │   │   │           ├── BC_h_2022_2100_cmip7.anc
    │   │   │           ├── Bio_h_2022_2100_cmip7.anc
    │   │   │           ├── OCFF_h_2022_2100_cmip7.anc
    │   │   │           └── scycl_h_2022_2100_cmip7.anc
    │   │   ├── forcing
    │   │   │   └── resolution_independent
    │   │   │       └── 2026.06.15
    │   │   │           ├── TSI_CMIP7_ESM
    │   │   │           └── volcts_cmip7.dat
    │   │   └── land
    │   │       └── biogeochemistry
    │   │           └── global.N96
    │   │               └── 2026.06.15
    │   │                   └── Ndep_h_2022_2100_cmip7.anc
    │   └── forcing
    │       └── global.N96
    │           └── 2026.06.15
    │               └── ozone_h_2022_2100_cmip7.anc
    └── vl
        ├── atmosphere
        │   ├── aerosol
        │   │   └── global.N96
        │   │       └── 2026.06.15
        │   │           ├── BC_vl_2022_2100_cmip7.anc
        │   │           ├── Bio_vl_2022_2100_cmip7.anc
        │   │           ├── OCFF_vl_2022_2100_cmip7.anc
        │   │           └── scycl_vl_2022_2100_cmip7.anc
        │   ├── forcing
        │   │   └── resolution_independent
        │   │       └── 2026.06.15
        │   │           ├── TSI_CMIP7_ESM
        │   │           └── volcts_cmip7.dat
        │   └── land
        │       └── biogeochemistry
        │           └── global.N96
        │               └── 2026.06.15
        │                   └── Ndep_vl_2022_2100_cmip7.anc
        └── forcing
            └── global.N96
                └── 2026.06.15
                    └── ozone_vl_2022_2100_cmip7.anc

59 directories, 30 files

More compactly:

$ cd /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.15/ && find . -type f | sort
./modern/historical/atmosphere/aerosol/global.N96/2026.06.15/BC_1849_2023_cmip7.anc
./modern/historical/atmosphere/aerosol/global.N96/2026.06.15/Bio_1849_2023_cmip7.anc
./modern/historical/atmosphere/aerosol/global.N96/2026.06.15/OCFF_1849_2023_cmip7.anc
./modern/historical/atmosphere/aerosol/global.N96/2026.06.15/scycl_1849_2023_cmip7.anc
./modern/historical/atmosphere/forcing/resolution_independent/2026.06.15/TSI_CMIP7_ESM
./modern/historical/atmosphere/forcing/resolution_independent/2026.06.15/volcts_cmip7.dat
./modern/historical/atmosphere/land/biogeochemistry/global.N96/2026.06.15/Ndep_1849_2023_cmip7.anc
./modern/historical/forcing/global.N96/2026.06.15/ozone_1849_2023_cmip7.anc
./modern/pre-industrial/atmosphere/aerosol/global.N96/2026.06.15/BC_1850_cmip7.anc
./modern/pre-industrial/atmosphere/aerosol/global.N96/2026.06.15/Bio_1850_cmip7.anc
./modern/pre-industrial/atmosphere/aerosol/global.N96/2026.06.15/OCFF_1850_cmip7.anc
./modern/pre-industrial/atmosphere/aerosol/global.N96/2026.06.15/scycl_1850_cmip7.anc
./modern/pre-industrial/atmosphere/land/biogeochemistry/global.N96/2026.06.15/Ndep_1850_cmip7.anc
./modern/pre-industrial/forcing/global.N96/2026.06.15/ozone_1850_cmip7.anc
./scenarios/h/atmosphere/aerosol/global.N96/2026.06.15/BC_h_2022_2100_cmip7.anc
./scenarios/h/atmosphere/aerosol/global.N96/2026.06.15/Bio_h_2022_2100_cmip7.anc
./scenarios/h/atmosphere/aerosol/global.N96/2026.06.15/OCFF_h_2022_2100_cmip7.anc
./scenarios/h/atmosphere/aerosol/global.N96/2026.06.15/scycl_h_2022_2100_cmip7.anc
./scenarios/h/atmosphere/forcing/resolution_independent/2026.06.15/TSI_CMIP7_ESM
./scenarios/h/atmosphere/forcing/resolution_independent/2026.06.15/volcts_cmip7.dat
./scenarios/h/atmosphere/land/biogeochemistry/global.N96/2026.06.15/Ndep_h_2022_2100_cmip7.anc
./scenarios/h/forcing/global.N96/2026.06.15/ozone_h_2022_2100_cmip7.anc
./scenarios/vl/atmosphere/aerosol/global.N96/2026.06.15/BC_vl_2022_2100_cmip7.anc
./scenarios/vl/atmosphere/aerosol/global.N96/2026.06.15/Bio_vl_2022_2100_cmip7.anc
./scenarios/vl/atmosphere/aerosol/global.N96/2026.06.15/OCFF_vl_2022_2100_cmip7.anc
./scenarios/vl/atmosphere/aerosol/global.N96/2026.06.15/scycl_vl_2022_2100_cmip7.anc
./scenarios/vl/atmosphere/forcing/resolution_independent/2026.06.15/TSI_CMIP7_ESM
./scenarios/vl/atmosphere/forcing/resolution_independent/2026.06.15/volcts_cmip7.dat
./scenarios/vl/atmosphere/land/biogeochemistry/global.N96/2026.06.15/Ndep_vl_2022_2100_cmip7.anc
./scenarios/vl/forcing/global.N96/2026.06.15/ozone_vl_2022_2100_cmip7.anc

penguian · 2026-06-15T12:02:24Z

Please note:

The Python code for the ScenarioMIP aerosol forcings is AI-assisted and partially based on code generated by Gemini 3.5 Flash via Google Antigravity.
My testing of the ScenarioMIP IIASA-IAMC aerosol forcings discovered that some of the datasets are not contiguous in time. Had I been more familiar with the CMIP6 scenario forcings this would not have been a surprise.
See Document the non-contiguous IIASA-IAMC ScenarioMIP datasets PCMDI/input4MIPs_CVs#447

penguian · 2026-06-17T03:01:16Z

I have changed the ScenarioMIP start year back to 2022 for all except greenhouse gases.

The run output is at /scratch/tm70/pcl851/cylc-run/u-dq819.scenarios/run82/

The new branches are

The file output is identical to the run on 2026-07-15, except for the ScenarioMIP aerosol forcings, and is at:

$ cd /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.17/ && find . -type f | sort
./modern/historical/atmosphere/aerosol/global.N96/2026.06.17/BC_1849_2023_cmip7.anc
./modern/historical/atmosphere/aerosol/global.N96/2026.06.17/Bio_1849_2023_cmip7.anc
./modern/historical/atmosphere/aerosol/global.N96/2026.06.17/OCFF_1849_2023_cmip7.anc
./modern/historical/atmosphere/aerosol/global.N96/2026.06.17/scycl_1849_2023_cmip7.anc
./modern/historical/atmosphere/forcing/resolution_independent/2026.06.17/TSI_CMIP7_ESM
./modern/historical/atmosphere/forcing/resolution_independent/2026.06.17/volcts_cmip7.dat
./modern/historical/atmosphere/land/biogeochemistry/global.N96/2026.06.17/Ndep_1849_2023_cmip7.anc
./modern/historical/forcing/global.N96/2026.06.17/ozone_1849_2023_cmip7.anc
./modern/pre-industrial/atmosphere/aerosol/global.N96/2026.06.17/BC_1850_cmip7.anc
./modern/pre-industrial/atmosphere/aerosol/global.N96/2026.06.17/Bio_1850_cmip7.anc
./modern/pre-industrial/atmosphere/aerosol/global.N96/2026.06.17/OCFF_1850_cmip7.anc
./modern/pre-industrial/atmosphere/aerosol/global.N96/2026.06.17/scycl_1850_cmip7.anc
./modern/pre-industrial/atmosphere/land/biogeochemistry/global.N96/2026.06.17/Ndep_1850_cmip7.anc
./modern/pre-industrial/forcing/global.N96/2026.06.17/ozone_1850_cmip7.anc
./scenarios/h/atmosphere/aerosol/global.N96/2026.06.17/BC_h_2022_2100_cmip7.anc
./scenarios/h/atmosphere/aerosol/global.N96/2026.06.17/Bio_h_2022_2100_cmip7.anc
./scenarios/h/atmosphere/aerosol/global.N96/2026.06.17/OCFF_h_2022_2100_cmip7.anc
./scenarios/h/atmosphere/aerosol/global.N96/2026.06.17/scycl_h_2022_2100_cmip7.anc
./scenarios/h/atmosphere/forcing/resolution_independent/2026.06.17/TSI_CMIP7_ESM
./scenarios/h/atmosphere/forcing/resolution_independent/2026.06.17/volcts_cmip7.dat
./scenarios/h/atmosphere/land/biogeochemistry/global.N96/2026.06.17/Ndep_h_2022_2100_cmip7.anc
./scenarios/h/forcing/global.N96/2026.06.17/ozone_h_2022_2100_cmip7.anc
./scenarios/vl/atmosphere/aerosol/global.N96/2026.06.17/BC_vl_2022_2100_cmip7.anc
./scenarios/vl/atmosphere/aerosol/global.N96/2026.06.17/Bio_vl_2022_2100_cmip7.anc
./scenarios/vl/atmosphere/aerosol/global.N96/2026.06.17/OCFF_vl_2022_2100_cmip7.anc
./scenarios/vl/atmosphere/aerosol/global.N96/2026.06.17/scycl_vl_2022_2100_cmip7.anc
./scenarios/vl/atmosphere/forcing/resolution_independent/2026.06.17/TSI_CMIP7_ESM
./scenarios/vl/atmosphere/forcing/resolution_independent/2026.06.17/volcts_cmip7.dat
./scenarios/vl/atmosphere/land/biogeochemistry/global.N96/2026.06.17/Ndep_vl_2022_2100_cmip7.anc
./scenarios/vl/forcing/global.N96/2026.06.17/ozone_vl_2022_2100_cmip7.anc

penguian · 2026-06-17T03:08:44Z

The reason why there is no new historical config branch is that there are no changes from the existing branch:

$ cat /scratch/tm70/pcl851/cylc-run/u-dq819.scenarios/run82/log/job/1/HI_git_commit_push/NN/job.out
Workflow : u-dq819.scenarios/run82
Job : 1/HI_git_commit_push/01 (try 1)
User@Host: pcl851@cylc.pcl851.tm70.ps.gadi.nci.org.au

2026-06-17T10:22:36+10:00 INFO - started
git status is:

Finished with no config files changes to commit.
2026-06-17T10:22:38+10:00 INFO - succeeded

penguian · 2026-06-17T11:19:43Z

Based on feedback from @MartinDix I have changed the interpolation method so that each month in the year is interpolated separately.

The new run is at /scratch/tm70/pcl851/cylc-run/u-dq819.scenarios/run86

The run re-generated the ScenarioMIP ancillaries only.

The newly generated branches are:

The updated file output is:

[pcl851@gadi-login-02 scenarios]$ pwd
/g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.17/scenarios
[pcl851@gadi-login-02 scenarios]$ find . -type f | sort | xargs ls -lh-rw-r--r-- 1 pcl851 tm70 213M Jun 17 19:30 ./h/atmosphere/aerosol/global.N96/2026.06.17/BC_h_2022_2100_cmip7.anc
-rw-r--r-- 1 pcl851 tm70 422M Jun 17 19:29 ./h/atmosphere/aerosol/global.N96/2026.06.17/Bio_h_2022_2100_cmip7.anc
-rw-r--r-- 1 pcl851 tm70 213M Jun 17 19:29 ./h/atmosphere/aerosol/global.N96/2026.06.17/OCFF_h_2022_2100_cmip7.anc
-rw-r--r-- 1 pcl851 tm70 631M Jun 17 19:29 ./h/atmosphere/aerosol/global.N96/2026.06.17/scycl_h_2022_2100_cmip7.anc
-rw-r--r-- 1 pcl851 tm70 8.3K Jun 17 19:35 ./h/atmosphere/forcing/resolution_independent/2026.06.17/TSI_CMIP7_ESM
-rw-r--r-- 1 pcl851 tm70 125K Jun 17 19:41 ./h/atmosphere/forcing/resolution_independent/2026.06.17/volcts_cmip7.dat
-rw-r--r-- 1 pcl851 tm70 213M Jun 17 19:28 ./h/atmosphere/land/biogeochemistry/global.N96/2026.06.17/Ndep_h_2022_2100_cmip7.anc
-rw-r--r-- 1 pcl851 tm70 165M Jun 17 20:09 ./h/forcing/global.N96/2026.06.17/ozone_h_2022_2100_cmip7.anc
-rw-r--r-- 1 pcl851 tm70 213M Jun 17 19:29 ./vl/atmosphere/aerosol/global.N96/2026.06.17/BC_vl_2022_2100_cmip7.anc
-rw-r--r-- 1 pcl851 tm70 422M Jun 17 19:29 ./vl/atmosphere/aerosol/global.N96/2026.06.17/Bio_vl_2022_2100_cmip7.anc
-rw-r--r-- 1 pcl851 tm70 213M Jun 17 19:29 ./vl/atmosphere/aerosol/global.N96/2026.06.17/OCFF_vl_2022_2100_cmip7.anc
-rw-r--r-- 1 pcl851 tm70 631M Jun 17 19:29 ./vl/atmosphere/aerosol/global.N96/2026.06.17/scycl_vl_2022_2100_cmip7.anc
-rw-r--r-- 1 pcl851 tm70 8.3K Jun 17 19:42 ./vl/atmosphere/forcing/resolution_independent/2026.06.17/TSI_CMIP7_ESM
-rw-r--r-- 1 pcl851 tm70 125K Jun 17 19:42 ./vl/atmosphere/forcing/resolution_independent/2026.06.17/volcts_cmip7.dat
-rw-r--r-- 1 pcl851 tm70 213M Jun 17 19:28 ./vl/atmosphere/land/biogeochemistry/global.N96/2026.06.17/Ndep_vl_2022_2100_cmip7.anc
-rw-r--r-- 1 pcl851 tm70 165M Jun 17 20:08 ./vl/forcing/global.N96/2026.06.17/ozone_vl_2022_2100_cmip7.anc

penguian · 2026-06-23T02:11:31Z

Note: The change to the interpolate_monthly function used by ScenarioMIP is partially AI-generated via Google Antigravity and Gemini 3.5 Flash.

penguian · 2026-06-26T07:44:07Z

I force pushed because I accidentally added commits that are relevant only to #121 and then rebased the commits away.

penguian added 9 commits April 28, 2026 16:05

Add an activity arg to cmip7_nitrogen_dirpath

3836dc3

Generate ScenarioMIP nitrogen ancils

111be59

Generate Ozone ancillaries for ScenarioMIP

41d6117

Refactor volcanic forcings and add ScenarioMIP

3f7bcf5

Fix the call to taper_saod

6b7b5c3

Refactor solar forcings and add ScenarioMIP

c0e1ebe

Ensure that the solar save directory exists

838b776

Set pi_year_mean correctly

c13e8c9

Use the correct activity

99931c4

penguian self-assigned this Jun 2, 2026

penguian linked an issue Jun 2, 2026 that may be closed by this pull request

Produce nitrogen and ozone ancillaries for the ACCESS-ESM1.6 ScenarioMIP h and vl experiments #127

Open

penguian added 3 commits June 5, 2026 11:30

Refactor greenhouse gas forcings and add ScenarioMIP

dd1544f

Apply ruff formatting to satisfy pre-commit

8585ada

Apply ruff formatting to satisfy pre-commit

e5d360f

penguian changed the title ~~Python code for nitrogen, ozone, solar and volcanic forcings for ScenarioMIP for ACCESS-ESM1.6~~ Python code for greenhouse gas, nitrogen, ozone, solar and volcanic forcings for ScenarioMIP for ACCESS-ESM1.6 Jun 5, 2026

penguian requested a review from MartinDix June 9, 2026 04:13

penguian marked this pull request as draft June 9, 2026 04:14

penguian added 12 commits June 9, 2026 17:04

Refactor to use dirpath and filename

b39bcac

Add draft ScenarioMIP scripts in progress

5440203

Use PI DMS for ScenarioMIP SO2 interpolation

68230ff

Set dirpath correctly

41d94ab

Use --dataset-date-range

b165688

Consistently use dataset_date_range

2386c92

Add --scenario

e9607d9

In load_sector_dict, use the correct filepath

e423db3

Add debug prints

47e33c9

When calling save_ancil, set replace_bounds=True

78e425c

Add interpolate_monthly (AI assisted)

7de7faf

In call to DimCoord, copy more parameters from time_coord

7e0867d

penguian added 11 commits June 13, 2026 21:19

Revert the diagnostics in extend_years

1905719

Sum IIASA-IAMC sectors (partly AI generated)

d7abf90

Add and use the _biomass_variable function

eaf44e3

Correctly use _biomass_variable

500ee24

Use interpolate_monthly

b2f048a

Remove unused code and parameters

d54eb52

Remove unused arguments

1fd24a7

In zero_poles, make data writeable

7f9d6d3

Move the corrected zero_poles to cmip7_aerosol_common.py

2b89b75

Modify a copy of data then write it back

97c57c9

Try setting CMIP7_SM_BEG_YEAR to 2023

a947a4b

penguian changed the title ~~Python code for greenhouse gas, nitrogen, ozone, solar and volcanic forcings for ScenarioMIP for ACCESS-ESM1.6~~ Python code for aerosol, greenhouse gas, nitrogen, ozone, solar and volcanic forcings for ScenarioMIP for ACCESS-ESM1.6 Jun 15, 2026

Satisfy pre-commit

3933ba7

penguian marked this pull request as ready for review June 15, 2026 11:55

penguian requested a review from atteggiani June 15, 2026 11:55

penguian marked this pull request as draft June 16, 2026 23:30

penguian added 2 commits June 17, 2026 09:32

Try changing CMIP7_SM_BEG_YEAR back to 2022

8853f0b

Start ScenarioMIP greenhouse gas processing one year late

04cf8d4

penguian marked this pull request as ready for review June 17, 2026 03:08

penguian added 3 commits June 17, 2026 18:39

Interpolate each month separately

614a990

Tidy up for ruff

1b1bf8a

Refactor interpolate_monthly

57a0305

penguian force-pushed the 127-n2-and-o3-for-esm16-h-and-vl branch from 9cec57f to 57a0305 Compare June 26, 2026 07:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Python code for aerosol, greenhouse gas, nitrogen, ozone, solar and volcanic forcings for ScenarioMIP for ACCESS-ESM1.6 #135

Python code for aerosol, greenhouse gas, nitrogen, ozone, solar and volcanic forcings for ScenarioMIP for ACCESS-ESM1.6 #135
penguian wants to merge 50 commits into
mainfrom
127-n2-and-o3-for-esm16-h-and-vl

penguian commented Jun 2, 2026 •

edited

Loading

Uh oh!

penguian commented Jun 5, 2026

Uh oh!

penguian commented Jun 15, 2026

Uh oh!

penguian commented Jun 15, 2026

Uh oh!

penguian commented Jun 17, 2026

Uh oh!

penguian commented Jun 17, 2026

Uh oh!

penguian commented Jun 17, 2026 •

edited

Loading

Uh oh!

penguian commented Jun 23, 2026

Uh oh!

penguian commented Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

penguian commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

penguian commented Jun 5, 2026

Uh oh!

penguian commented Jun 15, 2026

Uh oh!

penguian commented Jun 15, 2026

Uh oh!

penguian commented Jun 17, 2026

Uh oh!

penguian commented Jun 17, 2026

Uh oh!

penguian commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

penguian commented Jun 23, 2026

Uh oh!

penguian commented Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

penguian commented Jun 2, 2026 •

edited

Loading

penguian commented Jun 17, 2026 •

edited

Loading