Skip to content

Commit caac92d

Browse files
ci: [Backport] Introduced vetting test on PRs and corrected package version to match "current state of the package" (#3550)
This PR is a backport of #3549 and introduces vetting test that should be running on PR to confirm that the package will be in "releasable state". Things to note are ## Bumping package version After N4E investigation in I noticed that vetting tests were not working properly because after each release we should bump package version to match "current state of the package" which means bumping it by a patch. That way the current package version is being compared against last released. Otherwise this test won't give us any valid result because we will be comparing 2 already released packages. Because of that I bumped NGO internal version to 2.5.0 and in that way we will be comparing this "current" version against last released 2.4.3 ## What this test will do This test will mostly evaluate API compatibility. What we want to achieve is to not release any new minor version if not needed/detect all new APIs earlier. Notice that this test will fail if any new API is introduced (because for new API we need to release new minor and not patch) and in such scenario we should either - Bump package version to new minor, indicating that we will release new minor version - Make the API internal/private This will ensure that we know the current state of the package version. Notice that in theory we could also use wrench/api-validation test but this would require manually modifying this file after every wrench update (to add release.py execution) so easier approach will be to create a dedicated test and leave wrench one for actual release validation ## release.py script This python script is basically copied from N4E repo and updates CHANGELOG and any other places that need to be formatted for the release. For now it's only used for this test (to avoid it throwing an error about CHANGELOG being improperly formatted) but in the future we can use it to automate releasing ## Backport #3549
1 parent 33bb5c1 commit caac92d

File tree

6 files changed

+134
-2
lines changed

6 files changed

+134
-2
lines changed

.yamato/_run-all.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ run_quick_checks:
1717
dependencies:
1818
- .yamato/package-pack.yml#package_pack_-_ngo_ubuntu
1919
- .yamato/project-standards.yml#standards_ubuntu_testproject_trunk
20+
# Run API validation to early-detect all new APIs that would force us to release new minor version of the package. Note that for this to work the package version in package.json must correspond to "actual package state" which means that it should be higher than last released version
21+
- .yamato/vetting-test.yml#vetting_test
2022

2123
# Runs all package tests
2224
run_all_package_tests:

.yamato/_triggers.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,8 @@ develop_nightly:
108108
# Build player for webgl platform on trunk and 2021 editors
109109
- .yamato/project-updated-dependencies-test.yml#updated-dependencies_testproject_NGO_ubuntu_trunk
110110
- .yamato/project-updated-dependencies-test.yml#updated-dependencies_testproject_NGO_win_2021.3
111+
# Run API validation to early-detect all new APIs that would force us to release new minor version of the package. Note that for this to work the package version in package.json must correspond to "actual package state" which means that it should be higher than last released version
112+
- .yamato/vetting-test.yml#vetting_test
111113

112114

113115
# Run all tests on weekly bases

.yamato/package-pack.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ package_pack_-_ngo_{{ platform.name }}:
3737
variables:
3838
XRAY_PROFILE: "supported ./pvpExceptions.json"
3939
commands:
40+
- python Tools/scripts/release.py # Needed to ensure that CHANGELOG is properly formatted for this test due to the fact that we have bumped package version (to properly perform vetting tests)
4041
- upm-pvp pack "com.unity.netcode.gameobjects" --output upm-ci~/packages
4142
- upm-pvp xray --packages "upm-ci~/packages/com.unity.netcode.gameobjects*.tgz" --results pvp-results
4243
- upm-pvp require {% if platform.name == "win" %}"%XRAY_PROFILE%"{% else %}"$XRAY_PROFILE"{% endif %} --results pvp-results --allow-missing

.yamato/project.metafile

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -161,8 +161,9 @@ validation_editors:
161161
- 6000.1
162162
- 6000.2
163163
- trunk
164-
165-
164+
minimal:
165+
- 2021.3
166+
166167
# Scripting backends used by Standalone RunTimeTests---------------------------------------------------
167168

168169
scripting_backends:

.yamato/vetting-test.yml

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
{% metadata_file .yamato/project.metafile %} # All configuration that is used to create different configurations (used in for loops) is taken from this file.
2+
---
3+
# DESCRIPTION--------------------------------------------------------------------------
4+
# This configuration defines vetting tests for the Tools package which allows to validate if the package is in releasable state. This is important in particular because of API validation that allows to detect if we are introducing any new APIs that will force us to bump package version to new minor
5+
# If this test fails with new API error we should either make those API internal OR bump package version to new minor (note that the package version reflects the current package state)
6+
7+
# Note that we are packing the package only (no project context) so if package have any soft dependencies then project should be used to test it (to enable those APIs)
8+
{% for editor in validation_editors.minimal -%}
9+
vetting_test:
10+
name: MP Tools - Vetting Test (Win, {{editor}} LTS)
11+
agent: { type: Unity::VM, flavor: b1.large, image: package-ci/win11:v4 }
12+
commands:
13+
- python Tools/scripts/release.py # Needed to ensure that CHANGELOG is properly formatted for this test
14+
- npm install -g "upm-ci-utils@stable" --registry https://artifactory.prd.it.unity3d.com/artifactory/api/npm/upm-npm
15+
- unity-downloader-cli --fast --wait --unity-version {{ editor }} --components editor --arch x64
16+
- upm-ci package pack --package-path com.unity.netcode.gameobjects
17+
- upm-ci package test -u .Editor --package-path com.unity.netcode.gameobjects --type vetting-tests
18+
artifacts:
19+
logs:
20+
paths:
21+
- pvp-results/*
22+
- test-results/**
23+
- upm-ci~/test-results/**
24+
- upm-ci~/upm-ci.log
25+
{% endfor -%}

Tools/scripts/release.py

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
"""
2+
This python script makes the NGO package release ready. What it does is:
3+
1) Update changelogs
4+
2) Update validation exception file based on manifest version
5+
6+
Note that this script NEEDS TO BE RUN FROM THE ROOT of the project.
7+
"""
8+
#!/usr/bin/env python3
9+
import datetime
10+
import json
11+
import os
12+
import re
13+
import subprocess
14+
import platform
15+
16+
package_name = 'com.unity.netcode.gameobjects'
17+
18+
def update_changelog(new_version):
19+
"""
20+
Cleans the [Unreleased] section of the changelog by removing empty subsections,
21+
then replaces the '[Unreleased]' tag with the new version and release date.
22+
"""
23+
24+
changelog_entry = f'## [{new_version}] - {datetime.date.today().isoformat()}'
25+
changelog_path = f'{package_name}/CHANGELOG.md'
26+
print("Latest CHANGELOG entry will be modified to: " + changelog_entry)
27+
28+
with open(changelog_path, 'r', encoding='UTF-8') as f:
29+
changelog_text = f.read()
30+
31+
# This pattern finds a line starting with '###', followed by its newline,
32+
# and then two more lines that contain only whitespace.
33+
# The re.MULTILINE flag allows '^' to match the start of each line.
34+
pattern = re.compile(r"^###.*\n\n\n", re.MULTILINE)
35+
36+
# Replace every match with an empty string. The goal is to remove empty CHANGELOG subsections.
37+
cleaned_content = pattern.sub('', changelog_text)
38+
39+
# Replace the [Unreleased] section with the new version + cleaned subsections
40+
changelog_text = re.sub(r'## \[Unreleased\]', changelog_entry, cleaned_content)
41+
42+
# Write the changes
43+
with open(changelog_path, 'w', encoding='UTF-8', newline='\n') as file:
44+
file.write(changelog_text)
45+
46+
47+
def update_validation_exceptions(new_version):
48+
"""
49+
Updates the ValidationExceptions.json file with the new package version.
50+
"""
51+
52+
validation_file = f'{package_name}/ValidationExceptions.json'
53+
54+
# If files do not exist, exit
55+
if not os.path.exists(validation_file):
56+
return
57+
58+
# Update the PackageVersion in the exceptions
59+
with open(validation_file, 'rb') as f:
60+
json_text = f.read()
61+
data = json.loads(json_text)
62+
updated = False
63+
for exceptionElements in ["WarningExceptions", "ErrorExceptions"]:
64+
exceptions = data.get(exceptionElements)
65+
66+
if exceptions is not None:
67+
for exception in exceptions:
68+
if 'PackageVersion' in exception:
69+
exception['PackageVersion'] = new_version
70+
updated = True
71+
72+
# If no exceptions were updated, we do not need to write the file
73+
if not updated:
74+
return
75+
76+
with open(validation_file, 'w', encoding='UTF-8', newline='\n') as json_file:
77+
json.dump(data, json_file, ensure_ascii=False, indent=2)
78+
json_file.write("\n") # Add newline cause Py JSON does not
79+
print(f" updated `{validation_file}`")
80+
81+
82+
def get_manifest_json_version(filename):
83+
"""
84+
Reads the package.json file and returns the version specified in it.
85+
"""
86+
with open(filename, 'rb') as f:
87+
json_text = f.read()
88+
data = json.loads(json_text)
89+
90+
return data['version']
91+
92+
93+
if __name__ == '__main__':
94+
manifest_path = f'{package_name}/package.json'
95+
package_version = get_manifest_json_version(manifest_path)
96+
97+
# Update the ValidationExceptions.json file
98+
# with the new package version OR remove it if not a release branch
99+
update_validation_exceptions(package_version)
100+
# Clean the CHANGELOG and add latest entry
101+
update_changelog(package_version)

0 commit comments

Comments
 (0)