CHORE: refactor PR code coverage measurement - #542
Merged
Gaurav Sharma (bewithgaurav) merged 4 commits intoAug 7, 2026
Conversation
Pin the published coverage report to the py314-django60 Linux leg so the posted number is deterministic instead of a nondeterministic upload race across the matrix (which swings ~8.5 points on identical code). Instrument the testapp suite so mssql-django's own tests count toward coverage: wrap the testapp run in `coverage run --parallel-mode`, run the Django suite in parallel mode too, then `coverage combine` both before `coverage xml`. Narrow the include from `*mssql*` to `*/mssql/*` so it is segment-anchored and never pulls testapp files into the report. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot started reviewing on behalf of
Gaurav Sharma (bewithgaurav)
July 24, 2026 07:54
View session
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors CI code coverage measurement for pull requests to make the reported coverage value deterministic and to ensure coverage reflects both upstream Django tests and this repo’s testapp suite, without touching product code.
Changes:
- Run the
testappsuite undercoverage run --parallel-modeso repo-owned regression tests contribute to coverage. - Run Django’s
tests/runtests.pyundercoverage run --parallel-mode, thencoverage combinethetestapp+ Django suite data before generatingdjango/coverage.xml, narrowing include scope to*/mssql/*. - Publish code coverage results only from the
py314-django60matrix leg to prevent nondeterministic “last upload wins” behavior across the Linux matrix.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| tox.ini | Wraps the manage.py test run in coverage parallel mode so testapp contributes to coverage data. |
| test.sh | Runs Django suite coverage in parallel mode, combines coverage data from repo root + django/, and tightens include filtering to the mssql/ package segment. |
| azure-pipelines.yml | Pins PublishCodeCoverageResults to py314-django60 to avoid matrix upload races and stabilize the reported coverage number. |
Resolve conflict from the Linux-steps refactor on dev: the coverage-publish pin (PublishCodeCoverageResults@1 -> py314-django60 only) that #542 added inline moved into the shared azure-pipelines-steps-linux.yml template, where the task now lives. tox.ini + test.sh (coverage instrumentation of the testapp suite via --parallel-mode + coverage combine) merged cleanly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Jahnvi Thakkar (jahnvi480)
approved these changes
Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
two changes to how PR code coverage gets measured. no product code touched.
pin the published report to one matrix leg
every Linux leg runs test.sh, writes django/coverage.xml, and publishes to the same artifact path. so the posted number is whichever leg wins the upload race. across the matrix that swings ~8.5 points (70.7% on django 3.2 up to 79.2% on django 6.0) for identical code, and the Windows job publishes nothing. pinned PublishCodeCoverageResults to the py314-django60 leg so the number is deterministic and reflects the newest supported stack (most backend paths exercised).
count the testapp suite, not just Django's
today only tests/runtests.py runs under
coverage run.manage.py test(our own testapp suite) runs uninstrumented, so every regression test we add is invisible to the number. now:coverage run --parallel-mode--parallel-modetoo, thencoverage combinemerges both data sets beforecoverage xml*mssql*to*/mssql/*(segment-anchored, so a local checkout named mssql-django can't pull testapp files into the report)net effect: coverage reflects the mssql backend exercised by both suites combined, and it's stable run to run.
verified locally against sql2022: combine unions hits across the two runs (1534 union 1540 = 1556 covered), report contains only mssql/*.py, testapp files excluded.