Skip to content

[Flaky Test Handling] Drop scout from the report-flaky-tests pipeline name - #290465

Open
csr wants to merge 2 commits into
elastic:mainfrom
csr:rename-report-flaky-tests-pipeline
Open

[Flaky Test Handling] Drop scout from the report-flaky-tests pipeline name#290465
csr wants to merge 2 commits into
elastic:mainfrom
csr:rename-report-flaky-tests-pipeline

Conversation

@csr

@csr csr commented Sep 11, 2026

Copy link
Copy Markdown
Member

Summary

Removes scout from the name of the report-flaky-tests pipeline added in #289870. The report covers flaky tests from every framework (Scout, Jest, FTR, Cypress), so the name gave the wrong impression.

  • kibana / scout / report-flaky-testskibana / report-flaky-tests (slug kibana-report-flaky-tests)
  • .buildkite/pipeline-resource-definitions/kibana-scout-report-flaky-tests.ymlkibana-report-flaky-tests.yml
  • .buildkite/pipelines/scout_report_flaky_tests.ymlreport_flaky_tests.yml
  • .buildkite/scripts/steps/scout_report_flaky_tests.shreport_flaky_tests.sh
  • .github/CODEOWNERS: @elastic/appex-qa @elastic/kibana-operations own these files (and check_flaky_test_issues.sh, the second step added in [Flaky Test Handling] Add check_flaky_test_issues CLI to find existing failed-test issues for flaky suites #290095), so either team can approve pipeline changes

@csr
csr requested a review from a team as a code owner September 11, 2026 08:54
@csr csr added backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes reviewer:scout Agentic PR Scout test review v9.6.0 labels Sep 11, 2026
@infra-vault-gh-plugin-prod

Copy link
Copy Markdown
🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

  • Click to trigger kibana-pull-request for this PR!
  • Click to trigger kibana-deploy-project-from-pr for this PR!
  • Click to trigger kibana-deploy-cloud-from-pr for this PR!
  • Click to trigger kibana-entity-store-performance-from-pr for this PR!
  • Click to trigger kibana-storybooks-from-pr for this PR!

@csr csr changed the title [Flaky Test Handling] Drop scout from the report-flaky-tests pipeline name [Flaky Test Handling] Drop scout from the report-flaky-tests pipeline name Sep 11, 2026
@csr csr self-assigned this Sep 11, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One non-blocking note on the CODEOWNERS change: a new ownership entry references a Buildkite step script that doesn't exist in the repo yet. The pipeline/script/resource renames themselves are consistent — file paths, the pipeline slug, pipeline_file, sparse-checkout paths, and the locations.yml link are all updated together, and the renamed shell script has no lingering references to the old names.

Generated by Claude Reviewer for #290465 · claude · opus · 221.6 AIC · ⌖ 25.9 AIC · ⊞ 5.5K

Comment thread .github/CODEOWNERS
/.buildkite/pipeline-resource-definitions/kibana-report-flaky-tests.yml @elastic/appex-qa @elastic/kibana-operations
/.buildkite/pipelines/report_flaky_tests.yml @elastic/appex-qa @elastic/kibana-operations
/.buildkite/scripts/steps/report_flaky_tests.sh @elastic/appex-qa @elastic/kibana-operations
/.buildkite/scripts/steps/check_flaky_test_issues.sh @elastic/appex-qa @elastic/kibana-operations

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ownership entry points to a path that doesn't exist in the repo yet. Unlike the three entries above it (which map to files renamed in this PR), .buildkite/scripts/steps/check_flaky_test_issues.sh is present neither on main nor at this PR's head. The description attributes it to #290095, but that PR is still open (unmerged) and adds the node scripts/check_flaky_test_issues CLI — not this Buildkite step — so the rule is inert and matches nothing until a future PR adds the script.

Not CI-breaking (the CODEOWNERS check regenerates the file and verifies teams; both teams are valid), but the entry is dead until the file lands, and it will silently rot if the eventual script is named or located differently. Consider adding this line in the PR that actually introduces the file, or confirm the final path before landing it here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is correct - I will merge this PR after #290095 (which contains the new file) is in.

@kibanamachine

Copy link
Copy Markdown
Contributor

💚 Build Succeeded

Metrics [docs]

Unknown metric groups

warm start memory

id before after diff
post forced gc heap baseline - 839815028 +839815028
post forced gc heap delta - -1939024 -1939024
post forced gc heap delta standard deviation - 1471591 +1471591
post forced gc heap target - 837876004 +837876004
tail heap delta - 7797855 +7797855
total +1685021454

cc @csr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes reviewer:scout Agentic PR Scout test review v9.6.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants