You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
15
19
16
-
## Release automation
17
20
18
-
The FloPy release procedure is mostly automated with GitHub Actions in [`release.yml`](../.github/workflows/release.yml). There are a few manual steps that need to be performed, however:
21
+
## Initial steps
22
+
23
+
The FloPy release procedure is mostly automated with GitHub Actions in [`release.yml`](../.github/workflows/release.yml), but there are a few manual steps to complete first:
19
24
20
25
1. Update `usgsprograms.txt` in the [GitHub pymake repository](https://github.com/modflowpy/pymake) with the path to the new MODFLOW 6 release. Also update all other targets in `usgsprograms.txt` with the path to new releases.
21
26
22
-
2.Recompile all of the executables released on the [GitHub executables repository](https://github.com/MODFLOW-USGS/executables) using the `buildall.py` pymake script and Intel compilers for all operating systems.
27
+
2.Trigger a new release for the [executables repository](https://github.com/MODFLOW-USGS/executables), either via the GitHub Actions UI or GitHub CLI. See the executables repo's `DEVELOPER.md` for more information. Wait for the release to be published.
23
28
24
-
3. Update the README.md on the [GitHub executables repository](https://github.com/MODFLOW-USGS/executables) with the information in the `code.md` file created by the `buildall.py` pymake script.
29
+
3. Update the authors in `CITATION.cff` for the Software/Code citation for FloPy, if required.
30
+
25
31
26
-
4. Make a new release on the [GitHub executables repository](https://github.com/MODFLOW-USGS/executables) and add all of the operating system specific zip files containing the compiled executables (`linux.zip`, `mac.zip`, `win64.zip`, `win32.zip`). Publish the new release.
32
+
## Release procedure
27
33
28
-
5. Update the authors in `CITATION.cff` for the Software/Code citation for FloPy, if required.
29
-
30
-
31
-
Next, make a release branch from develop (*e.g.*`v3.3.6`). The branch name should be the version number with a `v` prefix. Pushing this branch to GitHub will trigger the release workflow, detailed below. If the branch name ends with `rc` it is considered a release candidate and the workflow is a dry run, stopping after updating version info & plugin classes and running tests and notebooks. If the branch name does not end with `rc` the release is considered approved and the full procedure runs.
34
+
Once initial tasks are completed, making a new release involves the following steps:
32
35
33
-
After updating version information, regenerating plugin classes, and generating a changelog, the approved release workflow creates a draft pull request from the release branch into `master`. Merging this pull request triggers another job to create a draft release. Promoting this draft release to a full release will trigger a final job to publish the release to PyPI and reset the `develop` branch from `master`, incrementing the patch version number on `develop`.
36
+
1. Release from `master` branch
37
+
2. Reinitialize `develop` branch
38
+
3. Publish the package to PyPI
39
+
4. Publish the package to Conda
34
40
41
+
These steps are automated with GitHub Actions in [`release.yml`](../.github/workflows/release.yml), but documentation for running the steps manually is provided in the [Manual releases](#manual-releases) section below.
35
42
36
-
##Release procedure
43
+
### Automated releases
37
44
38
-
This procedure runs automatically in `release.yml` after a release branch is pushed to GitHub, except for the final step (updating the `conda-forge/flopy-feedstock` repository — there is a bot which will [automatically detect changes and create a PR](https://github.com/conda-forge/flopy-feedstock/pull/47) to do so).
45
+
To begin an automated release, create a release branch from `develop`. The release branch name should be the version number with a `v` prefix (e.g. `v3.3.6`), with an optional `rc` suffix. If the branch name ends with `rc` it is considered a preliminary release candidate. If the branch name does not end with `rc` the release is considered production-ready. The automated workflow for approved, production-ready releases includes more steps than for release candidates.
39
46
47
+
Pushing the release branch starts the automated workflow. Manual intervention is only necessary to approve autogenerated pull requests and publish the autogenerated release post.
40
48
41
-
### Release from master branch
49
+
When the release branch is pushed to the `modflowpy/flopy` repository, the release workflow begins with the following steps:
42
50
43
-
- Update MODFLOW 6 dfn files in the repository and MODFLOW 6 package classes by running `python -c 'import flopy; flopy.mf6.utils.generate_classes(branch="master", backup=False)'`
44
-
45
-
- Run `isort` and `black` on the updated MODFLOW 6 package classes. This can be achieved by running `python scripts/pull_request_prepare.py` from the project root. The commands `isort .` and `black .` can also be run individually instead.
51
+
- update version strings to match the version number in the release branch name
52
+
- regenerate plugin classes from MODFLOW 6 DFN files
53
+
- rerun tests and notebooks
54
+
- generate and update changelogs
55
+
- build and test the FloPy package
56
+
57
+
For more detail regarding specific steps, see the [Manual releases](#manual-releases) section below.
58
+
59
+
#### Dry runs
60
+
61
+
If the branch name ends with `rc`, the workflow is a dry run, and ends here. No PRs are opened, a release is not created, and the release is not published to PyPI or Conda. The `flopy` package and the changelog are uploaded as artifacts, allowing manual inspection.
62
+
63
+
#### Production mode
64
+
65
+
If the branch name does not end with `rc`, the workflow will proceed to open a PR updating `master` from the release branch. Merging this PR to `master` triggers another job to draft a release.
66
+
67
+
**Note:** the PR should be merged, not squashed. Squashing removes the commit history from the `master` branch and causes `develop` and `master` to diverge, which can cause future PRs updating `master` to replay commits from previous releases.
68
+
69
+
Publishing the release triggers jobs to publish the `flopy` package to PyPI and open a PR updating `develop` from `master`. This PR also updates version strings, incrementing the patch version number.
70
+
71
+
72
+
### Manual releases
73
+
74
+
As described above, making a release manually involves the following steps:
75
+
76
+
1. Release from `master` branch
77
+
2. Reinitialize `develop` branch
78
+
3. Publish the package to PyPI
79
+
4. Publish the package to Conda
80
+
81
+
82
+
#### Release from `master` branch
46
83
47
84
- Run `python scripts/update_version.py -v <semver>` to update the version number stored in `version.txt` and `flopy/version.py`. For an approved release use the `--approve` flag.
48
85
49
-
- Use `run_notebooks.py` in the `scripts` directory to rerun all of the notebooks in:
86
+
- Update MODFLOW 6 dfn files in the repository and MODFLOW 6 package classes by running `python -c 'import flopy; flopy.mf6.utils.generate_classes(branch="master", backup=False)'`
87
+
88
+
- Run `isort` and `black` on the `flopy` module. This can be achieved by running `python scripts/pull_request_prepare.py` from the project root. The commands `isort .` and `black .` can also be run individually instead.
89
+
90
+
- Use `run_notebooks.py` in the `scripts` directory to rerun all notebooks in:
- Generate a changelog with [git cliff](https://github.com/orhun/git-cliff): `git cliff --unreleased --tag=<version number>`.
56
-
57
-
- Commit the changes to the release branch and push the commit to the [upstream GitHub repository](https://github.com/modflowpy/flopy).
96
+
- Generate a changelog starting from the last release with [git cliff](https://github.com/orhun/git-cliff), for instance: `git cliff --config cliff.toml --unreleased --tag=<release version number>`.
58
97
59
-
-Build and check the package with:
98
+
-Prepend the release changelog to the comprehensive changelog file `docs/version_changes.md`.
60
99
61
-
```shell
62
-
python -m build
63
-
twine check --strict dist/*
64
-
```
100
+
- Commit changes to the release branch and push the commit to the [`modflowpy/flopy` GitHub repository](https://github.com/modflowpy/flopy) (you must have write permissions on the repo).
65
101
66
102
- Update master branch from the release branch, e.g. by opening and merging a pull request into `master`. The pull request should be merged, *not* squashed, in order to preserve the project's commit history.
67
103
68
-
-Tag the merge commit to`master`with the version number. Don't forget to commit the tag. Push the commit and tag to GitHub.
104
+
-[Make a release](https://github.com/modflowpy/flopy/releases) from the`master`branch. Add the release changelog as the release post contents. Publish the release.
69
105
70
-
-Make a release on [GitHub website](https://github.com/modflowpy/flopy/releases). Add version changes for [current release](https://github.com/modflowpy/flopy/blob/develop/docs/version_changes.md) from to release text. Publish release.
106
+
-GitHub should have tagged the most recent revision of `master` with the release number — if for some reason this did not occur automatically, tag `master` with the version number (`git tag <version>`) and push the tag to `modflowpy/flopy` (e.g., if you have your fork's remote named `origin` and `modflowpy/flopy` as `upstream`, as is typical, use `git push upstream --tags`).
71
107
72
108
73
-
### Reinitialize develop branch
109
+
####Reinitialize `develop` branch
74
110
75
111
1. Merge the `master` branch into the `develop` branch.
76
112
77
113
2. Set the version as appropriate: `python scripts/update_version.py -v <semver>`.
1. Make sure the latest `build` and `twine` tools are installed using:
87
125
@@ -103,7 +141,11 @@ twine check --strict dist/*
103
141
twine upload dist/*
104
142
```
105
143
106
-
#### Conda forge
144
+
##### Conda
145
+
146
+
For the Conda distribution, there [is a bot](https://github.com/regro-cf-autotick-bot) which will [automatically detect new package versions uploaded to PyPI and create a PR](https://github.com/conda-forge/flopy-feedstock/pull/50) to update the `conda-forge/flopy-feedstock` repository. This PR can be reviewed, updated if needed, and merged to update the package on the `conda-forge` channel.
147
+
148
+
A Conda update can be performed manually with the following steps, substituting one's own fork of `conda-forge/flopy-feedstock`:
107
149
108
150
1. Download the `*.tar.gz` file for the just-created release from the [GitHub website](https://github.com/modflowpy/flopy/releases).
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
15
14
16
-
## `make-release.py`
15
+
## Processing benchmarks
17
16
18
-
TODO
19
-
20
-
## `process_benchmarks.py`
21
-
22
-
The `process_benchmarks.py` script consumes JSON files created by `pytest-benchmark` and creates a CSV file containing benchmark results. The script takes 2 positional arguments:
17
+
The `process_benchmarks.py` script converts one or more JSON files created by [`pytest-benchmark`](https://pytest-benchmark.readthedocs.io/en/latest/) into a single CSV file containing benchmark results. The script takes 2 positional arguments:
23
18
24
19
1. The path to the directory containing the JSON files.
25
20
2. The path to the directory to create the results CSV.
26
21
27
-
## `pull_request_prepare.py`
22
+
Input JSON files are expected to be named according to the `pytest-benchmark` format used when the `--benchmark-autosave` is provided:
28
23
29
-
The `pull_request_prepare.py` script lints Python source code files by running `black` and `isort` on the `flopy` subdirectory. This script should be run before opening a pull request, as CI will fail if the code is not properly formatted.
24
+
```shell
25
+
<commitid>_<date>_<time>_<isdirty>.json
26
+
```
30
27
31
-
## `run_notebooks.py`
28
+
For instance, `e689af57e7439b9005749d806248897ad550eab5_20150811_041632_uncommitted-changes.json`.
32
29
33
-
TODO
30
+
**Note**: the `process_benchmarks.py` script depends on `seaborn`, which is not included as a dependency in either `etc/environment.yml` or in any of the optional groups in `setup.cfg`, since this is the only place it is used in this repository.
34
31
35
-
## `update_version.py`
32
+
## Preparing for PRs
36
33
37
-
The `update_version.py` script can be used to update the FloPy version number. If the script is run with no argument, the version number is not changed, but an updated timestamp is written to the `flopy/version.py` file. To set the FloPy version number, use the `--version` (short `-v`) option, e.g.:
34
+
The `pull_request_prepare.py` script lints Python source code files by running `black` and `isort` on the `flopy` subdirectory. This script should be run before opening a pull request, as CI will fail if the code is not properly formatted. For instance, from the project root:
38
35
39
36
```shell
40
-
python scripts/update_version.py 3.3.6
41
-
```
37
+
python scripts/pull_request_prepare.py
38
+
```
39
+
40
+
## Running notebooks
41
+
42
+
The `run_notebooks.py` script runs notebooks located in subdirectories of the `examples` directory, currently including:
43
+
44
+
-`examples/Notebooks`
45
+
-`examples/groundwater_paper/Notebooks`
46
+
-`examples/FAQ`
47
+
48
+
Notebooks are run using `jupytext --from_ipynb --execute <notebook path>`. Note that notebooks are under version control, and running them with this script will produce large changesets in your working tree as execution metadata is updated. You will likely want to discard these changes before committing, e.g. with `git restore examples/Notebooks` (and likewise for other notebook folders). Care should be taken to make sure desired changes to notebook cells are not discarded however.
49
+
50
+
## Updating version
51
+
52
+
The `update_version.py` script can be used to update FloPy version numbers. Running the script first updates the version in `version.txt`, then propagates the change to various other places version strings or timestamps are embedded in the repository:
53
+
54
+
-`flopy/version.py`
55
+
-`flopy/DISCLAIMER.md`
56
+
-`CITATION.cff`
57
+
-`README.md`
58
+
-`docs/notebook_examples.md`
59
+
-`docs/PyPI_release.md`
60
+
61
+
The script acquires a file lock before writing to files to make sure only one process edits the files at any given time and prevent desynchronization.
62
+
63
+
If the script is run with no arguments, the version number is not changed, but updated timestamps are written. To set the version number, use the `--version` (short `-v`) option, e.g.:
64
+
65
+
```shell
66
+
python scripts/update_version.py -v 3.3.6
67
+
```
68
+
69
+
To get the current version number, use the `--get` flag:
70
+
71
+
```shell
72
+
python scripts/update_version.py
73
+
```
74
+
75
+
This simply returns the contents of `version.txt` and does not write any changes to the repository's files.
76
+
77
+
By default, the script assumes a local development version of FloPy. The `--approve` flag should be used prior to releasing a new FloPy version. This will alter the `DISCLAIMER.md` file, substituting wording to indicate the version is no longer preliminary but approved for official release. See [the release docs](../docs/make_release.md) for more information.
0 commit comments