Skip to content

Commit dcd4df4

Browse files
potiukclaude
andcommitted
Add breeze pr auto-triage command
Add a new Breeze CLI command that helps maintainers efficiently triage open PRs from non-collaborators that don't meet minimum quality criteria. The command fetches open PRs via GitHub GraphQL API with optimized chunked queries, runs deterministic CI checks (failures, merge conflicts, missing test workflows), optionally runs LLM-based quality assessment, and presents flagged PRs interactively for maintainer review with author profiles and contribution history. Key features: - Optimized GraphQL queries with chunking to avoid GitHub timeout errors - Deterministic CI failure detection with categorized fix instructions - LLM assessment via `claude` or `codex` CLI for content quality - Interactive review with Rich panels, clickable links, and author context - "maintainer-accepted" label to skip PRs on future runs - Workflow approval support for first-time contributor PRs awaiting CI runs - Merge conflict and commits-behind detection with rebase guidance Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent dda8d27 commit dcd4df4

26 files changed

+3019
-58
lines changed

contributing-docs/05_pull_requests.rst

Lines changed: 61 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
1+
contributing-docs/05_pull_requests.rst
22
.. Licensed to the Apache Software Foundation (ASF) under one
33
or more contributor license agreements. See the NOTICE file
44
distributed with this work for additional information
@@ -75,6 +75,15 @@ Pull Request guidelines
7575
Before you submit a Pull Request (PR) from your forked repo, check that it meets
7676
these guidelines:
7777

78+
- Start with **Draft**: Until you are sure that your PR passes all the quality checks and tests, keep it
79+
in **Draft** status. This will signal to maintainers that the PR is not yet ready
80+
for review and it will prevent maintainers from accidentally merging it before
81+
it's ready. Once you are sure that your PR is ready for review, you can mark it as
82+
"Ready for review" in the GitHub UI. Our regular check will convert all PRs from
83+
non-collaborators that do not pass our quality gates to Draft status, so if you see
84+
that your PR is in Draft status and you haven't set it to Draft. check the
85+
comments to see what needs to be fixed.
86+
7887
- Include tests, either as doctests, unit tests, or both, to your pull request.
7988

8089
The Airflow repo uses `GitHub Actions <https://help.github.com/en/actions>`_ to
@@ -147,6 +156,57 @@ these guidelines:
147156
- Adhere to guidelines for commit messages described in this `article <https://cbea.ms/git-commit/>`_.
148157
This makes the lives of those who come after you (and your future self) a lot easier.
149158

159+
.. _pull-request-quality-criteria:
160+
161+
Pull Request quality criteria
162+
-----------------------------
163+
164+
Every open PR must meet the following minimum quality criteria before maintainers will review it.
165+
PRs that do not meet these criteria may be automatically converted to **draft** status by project
166+
tooling, with a comment explaining what needs to be fixed.
167+
168+
1. **Descriptive title** — The PR title must clearly describe the change.
169+
Generic titles such as "Fix bug", "Update code", "Changes", single-word titles, or titles
170+
that only reference an issue number (e.g. "Fixes #12345") do not meet this bar.
171+
172+
2. **Meaningful description** — The PR body must contain a meaningful description of *what* the
173+
PR does and *why*. An empty body, a body consisting only of the PR template
174+
checkboxes/headers with no added text, or a body that merely repeats the title is not
175+
sufficient.
176+
177+
3. **Passing static checks** — The PR's static checks (pre-commit / ruff / mypy) must pass.
178+
You can run them locally with ``prek run --ref-from main`` before pushing.
179+
180+
4. **Gen-AI disclosure** — If the PR was created with the assistance of generative AI tools,
181+
the description must include a disclosure (see `Gen-AI Assisted contributions`_ below).
182+
183+
5. **Coherent changes** — The PR should contain related changes only. Completely unrelated
184+
changes bundled together will be flagged.
185+
186+
**What happens when a PR is converted to draft?**
187+
188+
- The comment informs you what you need to do.
189+
- Fix each issue, then mark the PR as "Ready for review" in the GitHub UI - but only after making sure that all the issues are fixed.
190+
- Maintainers will then proceed with a normal review.
191+
192+
Converting a PR to draft is **not** a rejection — it is an invitation to bring the PR up to
193+
the project's standards so that maintainer review time is spent productively.
194+
195+
**What happens when a PR is closed for quality violations?**
196+
197+
If a contributor has more than 3 open PRs that are flagged for quality issues, maintainers may
198+
choose to **close** the PR instead of converting it to draft. Closed PRs receive the
199+
``closed because of multiple quality violations`` label and a comment listing the violations.
200+
Contributors are welcome to open a new PR that addresses the issues listed in the comment.
201+
202+
**What happens when suspicious changes are detected?**
203+
204+
When maintainers review a PR's diff before approving CI workflow runs and determine that it
205+
contains suspicious changes (e.g. attempts to exfiltrate secrets, modify CI pipelines
206+
maliciously, or inject harmful code), **all open PRs by the same author** will be closed
207+
and labeled ``suspicious changes detected``. A comment is posted on each PR explaining that
208+
the closure was triggered by suspicious changes found in the flagged PR.
209+
150210
Gen-AI Assisted contributions
151211
-----------------------------
152212

dev/breeze/doc/11_issues_tasks.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -54,5 +54,5 @@ Example usage:
5454
5555
-----
5656

57-
Next step: Follow the `Advanced Breeze topics <12_advanced_breeze_topics.rst>`__ instructions to learn more
58-
about advanced Breeze topics and internals.
57+
Next step: Follow the `Pull request tasks <13_pr_tasks.rst>`__ instructions to learn how to
58+
manage GitHub pull requests with Breeze.

dev/breeze/doc/13_pr_tasks.rst

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
.. Licensed to the Apache Software Foundation (ASF) under one
2+
or more contributor license agreements. See the NOTICE file
3+
distributed with this work for additional information
4+
regarding copyright ownership. The ASF licenses this file
5+
to you under the Apache License, Version 2.0 (the
6+
"License"); you may not use this file except in compliance
7+
with the License. You may obtain a copy of the License at
8+
9+
.. http://www.apache.org/licenses/LICENSE-2.0
10+
11+
.. Unless required by applicable law or agreed to in writing,
12+
software distributed under the License is distributed on an
13+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
KIND, either express or implied. See the License for the
15+
specific language governing permissions and limitations
16+
under the License.
17+
18+
Pull request tasks
19+
------------------
20+
21+
There are Breeze commands that help maintainers manage GitHub pull requests for the Apache Airflow project.
22+
23+
Those are all of the available PR commands:
24+
25+
.. image:: ./images/output_pr.svg
26+
:target: https://raw.githubusercontent.com/apache/airflow/main/dev/breeze/doc/images/output_pr.svg
27+
:width: 100%
28+
:alt: Breeze PR commands
29+
30+
Auto-triaging PRs
31+
"""""""""""""""""
32+
33+
The ``breeze pr auto-triage`` command finds open PRs from non-collaborators that don't meet
34+
minimum quality criteria and lets maintainers take action on them interactively.
35+
36+
.. image:: ./images/output_pr_auto-triage.svg
37+
:target: https://raw.githubusercontent.com/apache/airflow/main/dev/breeze/doc/images/output_pr_auto-triage.svg
38+
:width: 100%
39+
:alt: Breeze PR auto-triage
40+
41+
The command works in several phases:
42+
43+
1. **Fetch** — Fetches open, non-draft PRs via the GitHub GraphQL API.
44+
2. **Filter** — Skips collaborators, bot accounts (dependabot, renovate, github-actions),
45+
and PRs already labeled ``ready for maintainer review``.
46+
3. **Assess** — Runs deterministic checks (CI failures, merge conflicts, missing test
47+
workflows) and optionally LLM-based quality assessment (via ``claude`` or ``codex`` CLI).
48+
4. **Triage** — Presents flagged PRs grouped by author. For each PR the maintainer chooses
49+
an action:
50+
51+
* **[D]raft** — Convert to draft and post a comment listing the violations (default for
52+
most PRs).
53+
* **[C]lose** — Close the PR, add the ``closed because of multiple quality violations``
54+
label, and post a comment. This is the suggested default when the author has more than
55+
3 flagged PRs.
56+
* **[R]eady** — Add the ``ready for maintainer review`` label so the PR is skipped in
57+
future runs.
58+
* **[S]kip** — Take no action on this PR.
59+
* **[Q]uit** — Stop processing.
60+
61+
The command computes a smart default action based on CI failures, merge conflicts, LLM
62+
assessment, and the number of flagged PRs by the same author. In ``--dry-run`` mode the
63+
default action is displayed without prompting.
64+
65+
5. **Workflow approval** — PRs with no test workflows run are presented for workflow
66+
approval. Before approving, the maintainer reviews the full PR diff to check for
67+
suspicious changes (e.g. attempts to exfiltrate secrets or modify CI pipelines). If
68+
suspicious changes are confirmed, **all open PRs by that author** are closed, labeled
69+
``suspicious changes detected``, and commented.
70+
71+
Example usage:
72+
73+
.. code-block:: bash
74+
75+
# Dry run to see which PRs would be flagged and what action would be taken
76+
breeze pr auto-triage --dry-run
77+
78+
# Run with CI checks only (no LLM)
79+
breeze pr auto-triage --check-mode ci
80+
81+
# Filter by label and author
82+
breeze pr auto-triage --label area:core --author some-user
83+
84+
# Limit to 10 PRs
85+
breeze pr auto-triage --max-num 10
86+
87+
# Verbose mode — show individual skip reasons during filtering
88+
breeze pr auto-triage --verbose

dev/breeze/doc/README.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ The following documents describe how to use the Breeze environment:
5050
* `Release management tasks <09_release_management_tasks.rst>`_ - describes how to use Breeze for release management tasks.
5151
* `UI tasks <10_ui_tasks.rst>`_ - describes how Breeze commands are used to support Apache Airflow project UI.
5252
* `Issues tasks <11_issues_tasks.rst>`_ - describes how Breeze commands are used to manage GitHub issues.
53+
* `Pull request tasks <13_pr_tasks.rst>`_ - describes how Breeze commands are used to manage GitHub pull requests.
5354
* `Advanced Breeze topics <12_advanced_breeze_topics.rst>`_ - describes advanced Breeze topics/internals of Breeze.
5455

5556
You can also learn more context and Architecture Decisions taken when developing Breeze in the

0 commit comments

Comments
 (0)