Promotion 2025-09-30 prod (#7450)

[dsotirho-ucsc]

Linked issue: #7450

Note

• For Disjunctivity constraint enforced too late #7398, a reindex is required of the most recent non-IT catalog in each atlas for each deployment:
  • prod: dcp53 & lm9

Checklist

Author

• PR is assigned to the author
• Target branch is prod
• Name of PR branch matches promotions/yyyy-mm-dd-prod
• PR is linked to the promotion issue it resolves
• PR description links to connected issue
• Title of linked issue matches Promotion yyyy-mm-dd
• PR title starts with title of linked issue followed by prod
• PR title references the linked issue

Author (reindex)

• This PR is labeled reindex:prod _{or the changes introduced by it will not require reindexing of prod}
• This PR is labeled reindex:partial and its description documents the specific reindexing procedure for prod _{or requires a full reindex or is not labeledreindex:prod}

Author (upgrading deployments)

• This PR is labeled upgrade _{or does not require upgrading deployments}
• This PR is labeled deploy:shared _{or does not modify docker_images.json, and does not require deploying the shared component for any other reason}
• This PR is labeled deploy:gitlab _{or does not require deploying the gitlab component}
• This PR is labeled deploy:runner _{or does not require deploying the runner image}

Author (before every review)

• PR branch is up to date (if not, merge prod into PR branch to integrate upstream changes)
• PR is not a draft
• PR is awaiting requested review from system administrator
• Status of PR is Review requested
• PR is assigned to only the system administrator

System administrator (after approval)

• Actually approved the PR
• Labeled PR as no sandbox
• Status of PR is Approved
• PR is assigned to only the operator

Operator

• Pushed PR branch to GitHub

Operator (deploy .shared and .gitlab components)

• Ran _select prod.shared && CI_COMMIT_REF_NAME=prod make -C terraform/shared apply_keep_unused _{or this PR is not labeled deploy:shared}
• Ran _select prod.gitlab && python scripts/create_gitlab_snapshot.py --no-restart (see operator manual for details) _{or this PR is not labeled backup:gitlab}
• Ran _select prod.gitlab && CI_COMMIT_REF_NAME=prod make -C terraform/gitlab apply _{or this PR is not labeled deploy:gitlab}
• Checked the items in the next section _{or this PR is labeled deploy:gitlab}
• PR is assigned to only the system administrator _{or this PR is not labeled deploy:gitlab}

System administrator (post-deploy of .gitlab component)

• Background migrations for prod.gitlab are complete _{or this PR is not labeled deploy:gitlab}
• PR is assigned to only the operator

Operator (deploy runner image)

• Ran _select prod.gitlab && make -C terraform/gitlab/runner _{or this PR is not labeled deploy:runner}

Operator (merge the branch)

• All status checks passed and the PR is mergeable
• The title of the merge commit starts with the title of this PR
• Added PR # reference to merge commit title
• Collected commit title tags in merge commit title _{but excluded any p tags}
• Pushed merge commit to GitHub
• Status of PR is Merged stable

Operator (main build)

• Pushed merge commit to GitLab prod
• Build passes on GitLab prod
• Reviewed build logs for anomalies on GitLab prod
• Ran _select prod.shared && make -C terraform/shared apply _{or this PR is not labeled deploy:shared}
• Deleted PR branch from GitHub
• Status of linked issue is Stable
• Status of promoted¹ PRs is Merged stable
• Status of promoted¹ issues is Stable

¹ Promoted issues and PRs are referenced in the titles of the commits
that the promotion branch introduces to the stable branch. Prior to the
promotion, the status of promoted issues (PRs) is Lower (Merged lower).
Promoted PRs in status Done do not need to be moved.

Operator (reindex)

• Deindexed all unreferenced catalogs in prod _{or this PR is neither labeled reindex:partial nor reindex:prod}
• Deindexed specific sources in prod _{or this PR is neither labeled reindex:partial nor reindex:prod}
• Indexed specific sources in prod _{or this PR is neither labeled reindex:partial nor reindex:prod}
• Started reindex in prod _{or this PR does not require reindexing prod}
• Reindex of dcp53 completed
• Reindex of lm9 completed
• Checked for, triaged and possibly requeued messages in both fail queues in prod _{or this PR does not require reindexing prod}
• Emptied fail queues in prod _{or this PR does not require reindexing prod}
• Restarted the Data Browser pipeline for the ucsc/hca/prod branch on GitLab in prod _{or this PR does not require reindexing prod}
• Restarted the Data Browser pipeline for the ucsc/lungmap/prod branch on GitLab in prod _{or this PR does not require reindexing prod}
• Restarted deploy_browser job in the GitLab pipeline for this PR in prod _{or this PR does not require reindexing prod}

Operator (mirroring)

• Started mirroring in prod _{or this PR does not require mirroring prod}
• Checked for, triaged and possibly requeued messages in mirror fail queue in prod _{or this PR does not require mirroring prod}
• Emptied mirror fail queue in prod _{or this PR does not require mirroring prod}

Operator

• PR is assigned to only the system administrator

System administrator

• Removed unused image tags from pycharm image on DockerHub _{or this promotion does not alter references to that image}
• Removed unused image tags from bigquery_emulator image on DockerHub _{or this promotion does not alter references to that image}
• PR is assigned to no one

Shorthand for review comments

• L line is too long
• W line wrapping is wrong
• Q bad quotes
• F other formatting problem

[coveralls]

coverage: 85.365% (+0.03%) from 85.338%
when pulling 9a759b1 on promotions/2025-09-30-prod
into b6f8e4f on prod.

[codecov]

Codecov Report

❌ Patch coverage is 77.17391% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.17%. Comparing base (b6f8e4f) to head (9a759b1).
⚠️ Report is 69 commits behind head on prod.

Files with missing lines	Patch %	Lines
test/integration_test.py	0.00%	20 Missing ⚠️
src/azul/service/avro_pfb.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             prod    #7459      +/-   ##
==========================================
+ Coverage   85.14%   85.17%   +0.02%     
==========================================
  Files         156      155       -1     
  Lines       22278    22280       +2     
==========================================
+ Hits        18968    18976       +8     
+ Misses       3310     3304       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.