Skip to content

Changelog Generation with release_changelog #1249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
hkir-dev wants to merge 6 commits into
base: master
Choose a base branch
from
Draft

Changelog Generation with release_changelog #1249

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
fe42595
Fixes issue #470 – added release_changelog command to generate releas…
hkir-dev May 21, 2025
0b3d86c
Fixes issue #470 – using base owl artefact for the changelog
hkir-dev May 22, 2025
72c1d49
Fixes issue #470 – release_diff conversion disclaimer added
hkir-dev May 22, 2025
0a4094f
Fixes issue #470 – the step for removing owl-axioms has been removed …
hkir-dev May 22, 2025
c4b5a85
Fixes issue #470 – move mistakenly added command comment from release…
hkir-dev May 22, 2025
af4d57f
Fixes issue #470 – move mistakenly added command comment from release…
hkir-dev May 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/FrequentlyUsedODKCommands.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,8 @@ Example:
```
sh run.sh make validate_profile_hp-edit.owl
```
## Generate the release changelog

```
sh run.sh make release_changelog
```
23 changes: 23 additions & 0 deletions template/src/ontology/Makefile.jinja2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1556,6 +1556,28 @@ update_docs:
mkdocs gh-deploy --config-file ../../mkdocs.yaml
{%- endif %}

.PHONY: release_changelog
release_changelog: $(REPORTDIR)/changelog.md $(REPORTDIR)/changelog.csv $(REPORTDIR)/changelog.yaml
Copy link
Contributor

@gouttegd gouttegd May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open question: for now, the release changelogs are only generated “on demand”, when a user explicitly runs make release_changelog.

Do we want the changelog to be generated automatically? The original issue is uncommitted about that that.

(Personally I’d say no – i.e., leave things like that, generate on-demand only –, or at least, if it is generated automatically make it configurable.)

Copy link
Contributor

@StroemPhi StroemPhi May 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd favour to be able to activate it via the config yaml. Running it on demand only would suffice, though.

Copy link
Contributor

@matentzn matentzn May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My worry is if you do NOT fold it into the release process, then some quirk in the dependency chain might cause a complete regeneration of the "current release" locally, while its 99% certain that, if the rule is indeed run manually, the release has already been generated.

Also, if the temporary release files are copied up during prepare_release, it will definitely regenerate the release.

So I vote for

  1. making this configurable in odk.yaml
  2. including it in the normal release process
  3. While at (1), also add an option to configure the ontology variant used for the diff.

Copy link
Contributor

@gouttegd gouttegd May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My worry is if you do NOT fold it into the release process, then some quirk in the dependency chain might cause a complete regeneration of the "current release" locally

This will only happen if a source file has changed since the last time the release was built – and in that case, re-generating the release before making the diff is the expected behaviour.

If a dependency on $(ONT)-base.owl causes $(ONT)-base.owl to be rebuilt even though no source has changed, then it’s not a “quirk in the dependency chain”, it’s a full-on bug – one that should be fixed once it is reported.

Also, if the temporary release files are copied up during prepare_release, it will definitely regenerate the release.

I don’t see why?

Copy link
Contributor

@matentzn matentzn May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t see why?

See here:

ontology-development-kit/template/src/ontology/Makefile.jinja2

Lines 433 to 439 in d85794b

prepare_release: all_odk
rsync -R $(RELEASE_ASSETS) $(RELEASEDIR) &&\{% if project.sssom_mappingset_group is defined %}{% if project.sssom_mappingset_group.released_products is defined %}
mkdir -p $(RELEASEDIR)/mappings && cp -rf $(RELEASED_MAPPING_FILES) $(RELEASEDIR)/mappings &&\{% endif %}{% endif %}{% if project.use_dosdps %}
mkdir -p $(RELEASEDIR)/patterns && cp -rf $(PATTERN_RELEASE_FILES) $(RELEASEDIR)/patterns &&\{% endif %}
rm -f $(CLEANFILES) &&\
echo "Release files are now in $(RELEASEDIR) - now you should commit, push and make a release \
on your git hosting site such as GitHub or GitLab"

People run: prepare_release; clean will clean up main release files so they are no longer in src/ontology. Running the diff will force the base to be rebuild.

Copy link
Contributor

@gouttegd gouttegd May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I would consider a bug. prepare_release should not, in my opinion, clean files after a release.

What is even the point of doing that? It only ensures that the next time prepare_release is run, it would be from a clean state.

It would be much useful to ensure that intermediate files are cleaned up before running the release (that’s in fact what I always do: I never run make prepare_release, always make clean prepare_release).

Copy link
Contributor

@gouttegd gouttegd May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⬆️ Anyway, this above is a different issue, and one that I am reluctant to fix now, so close to release – this “cleaning after release“ has been the behaviour of prepare_release for a while, I don’t think it is urgent to change it (I do think it should be changed, just not now).

Copy link
Contributor

@gouttegd gouttegd May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

making this configurable in odk.yaml
[…]
While at (1), also add an option to configure the ontology variant used for the diff.

For simplicity, I’d favour a single option, that decides both whether a changelog is generated and the type of artefact from which it is computed (instead of one option to enable the changelog and another one from the type of artefact).

Something like

changelog_from: (base|full|simple|none)

where none is understood as disabling the production of the changelog.

Copy link
Contributor

@matentzn matentzn May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like

Fine by me!


$(TMPDIR)/latest_release.obo: $(TMPDIR)
wget $(ONTBASE)/$(ONT)-base.owl -O [email protected] || { \
Copy link
Contributor

@matentzn matentzn May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if the dependence on "base" here should be documented somewhere. Not everyone likes the idea of having a base, but everyone will want to generate release notes.. Can we make the "diff" ontology configurable, using base by default if and only if it is a configured release product, otherwise the main release?

echo "Latest release couldn't be found, creating an empty OBO file instead"; \
echo "format-version: 1.2" > [email protected]; \
}
$(ROBOT) convert -i [email protected] --check false -f obo $(OBO_FORMAT_OPTIONS) -o $@ && rm [email protected]

$(TMPDIR)/current_release.obo: $(ONT)-base.owl
Copy link
Contributor

@matentzn matentzn May 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is good as it is not at all certain the ontology does indeed have an OBO release.

$(ROBOT) convert -i $< --check false -f obo $(OBO_FORMAT_OPTIONS) -o $@

$(REPORTDIR)/changelog.md: $(TMPDIR)/latest_release.obo $(TMPDIR)/current_release.obo
runoak -i simpleobo:$< diff -X simpleobo:$(word 2, $^) -o $@ --output-type md

$(REPORTDIR)/changelog.csv: $(TMPDIR)/latest_release.obo $(TMPDIR)/current_release.obo
runoak -i simpleobo:$< diff -X simpleobo:$(word 2, $^) -o $@ --output-type csv --statistics --group-by-property oio:hasOBONamespace

$(REPORTDIR)/changelog.yaml: $(TMPDIR)/latest_release.obo $(TMPDIR)/current_release.obo
runoak -i simpleobo:$< diff -X simpleobo:$(word 2, $^) -o $@ --output-type yaml

# Note to future generations: computing the real path relative to the
# current directory is a way to ensure we only clean up directories that
# are located below the current directory, regardless of the contents of
Expand Down Expand Up @@ -1627,6 +1649,7 @@ Additional build commands (advanced users)
* all_assets: Build all assets
* show_assets: Print a list of all assets that would be build by the release pipeline
* all_mappings: Update all SSSOM mapping sets
* release_changelog: Generates a markdown changelog report summarizing ontology changes along with KGCL outputs. This function converts OWL ontologies to OBO format for diffing, which may result in information loss due to OBO's limited expressivity compared to OWL.

Additional QC commands (advanced users)
* robot_reports: Run all configured ROBOT reports
Expand Down