Skip to content

PYTHON-5389: Add Tags to each buildvariant and augment test skipping/tracking policy. #82

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jul 23, 2025

Conversation

Jibola
Copy link
Collaborator

@Jibola Jibola commented Jul 1, 2025

Summary

I've added in a new stipulation on each buildvariant to require a tag specifying a language. As well, I've added linters to abide by the naming conventions on buildvariant.names as well.

Changes in this PR

  • Introduced lint_config.py which checks each buildvariant has a tag of a defined language
  • Added a pre-commit hook to run lint_config.py specifically on the config.yml file.
  • Updated any buildvariant name without a language to now include the language in its buildvariant name
  • Updated the README.md to highlight the need for a tag field with a language value on every buildvariant.

Screenshot(s)

Below shows the code catching a missing tag on a buildvariant in the config.yaml

image

Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds tagging support to build variants and updates the test notification policy in the documentation.

  • Updated README.md to document the new tags field and its effect on Slack notifications.
  • Modified .evergreen/config.yml to add language tags for each build variant.

Reviewed Changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated no comments.

File Description
README.md Updated documentation to include explanation for the new "tags" field.
.evergreen/config.yml Added a "tags" property for each build variant to support language tagging.
Comments suppressed due to low confidence (1)

README.md:190

  • There is an inconsistency in the naming of the notification channel: 'dbx-ai-ml-testing-pipline-notifications' appears to be a typo compared to 'dbx-ai-ml-testing-pipeline-notifications-{language}'. Consider standardizing the channel names.
Tests are run periodically (nightly) and any failures will propagate into both the `dbx-ai-ml-testing-pipline-notifications` and `dbx-ai-ml-testing-pipeline-notifications-{language}` channel. Repo owners of this `ai-ml-testing-pipeline` library are required to join the `dbx-ai-ml-testing-pipeline-notifications`. Pipeline specific implementers must **at least** join `dbx-ai-ml-testing-pipline-notifications-{language}` (e.g. whomever implemented `langchain-js` must at least be a member of `dbx-ai-ml-testing-pipeline-notifications-js`).

@Jibola Jibola requested a review from baileympearson July 15, 2025 18:31
Copy link
Member

@blink1073 blink1073 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Jibola Jibola merged commit 16ec3d5 into main Jul 23, 2025
11 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants