Skip to content

Commit 8d2f1dd

Browse files
justin808claude
andauthored
Add PR comment command to trigger full CI suite and fix workflow issues (#1974)
## Summary This PR adds a `/run-skipped-ci` command that can be used in PR comments to trigger the complete CI suite, including tests that are normally skipped on PRs for faster feedback. It also fixes several workflow issues identified during development. ## What Changed **New GitHub Actions Workflows:** - `.github/workflows/run-skipped-ci.yml` - Listens for `/run-skipped-ci` comments and triggers all CI workflows with verification - `.github/workflows/pr-welcome-comment.yml` - Automatically comments on new PRs explaining the command **New Documentation:** - `.github/README.md` - Comprehensive documentation of all workflows and the comment command feature **Workflow Fixes:** - Fixed duplicate `if` conditions in `examples.yml` and `main.yml` that caused YAML parsing errors - Replaced inefficient `SKIP_MINIMUM_ON_PR` env var pattern with cleaner matrix-level `exclude` approach - Added workflow verification logic to confirm triggered workflows actually start - Improved error handling with detailed status reporting (verified/pending/failed workflows) ## How It Works 1. **Default PR behavior** - Runs subset of CI (latest Ruby/Node only) for fast feedback 2. **Full CI on demand** - Comment `/run-skipped-ci` on any PR to run: - Main test suite with all Ruby/Node version combinations - Example generator tests - Pro package integration tests - Pro package unit tests 3. **User feedback** - Bot reacts with 🚀, waits 5 seconds for workflows to queue, verifies they started, and posts detailed status ## Why This Approach By default, PRs run a limited CI suite to keep feedback loops fast: - Only latest versions (Ruby 3.4, Node 22, Shakapacker 9.3.0, React 19) - Skips minimum versions (Ruby 3.2, Node 20, Shakapacker 8.2.0, React 18) Before merging, we want to verify compatibility across all supported versions. This command makes it easy to run full CI without: - Waiting for merge to master - Manually triggering individual workflows - Modifying workflow files ## Technical Improvements ### Matrix Filtering Before: ```yaml env: SKIP_MINIMUM_ON_PR: ${{ matrix.dependency-level == 'minimum' && ... }} steps: - uses: actions/checkout@v4 if: env.SKIP_MINIMUM_ON_PR != 'true' # ❌ Step-level conditional ``` After: ```yaml matrix: include: [...] exclude: # ✅ Job-level filtering - more efficient - ruby-version: ${{ ... && '3.2' || '' }} dependency-level: ${{ ... && 'minimum' || '' }} ``` ### Workflow Verification The `/run-skipped-ci` command now: 1. Triggers all workflows via API 2. Waits 5 seconds for workflows to queue 3. Queries GitHub Actions API to verify workflows started 4. Reports detailed status showing which workflows are verified, pending, or failed ## Testing Notes **Important:** The comment-triggered workflow must be merged to `master` before it can be tested, because GitHub Actions only executes `issue_comment` workflows from the default branch (security feature). Once merged, test by: 1. Creating a test PR 2. Commenting `/run-skipped-ci` 3. Verifying all workflows trigger and status is reported accurately ## Inspiration Based on similar pattern from shakacode/react-webpack-rails-tutorial#687 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- Reviewable:start --> - - - This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/shakacode/react_on_rails/1974) <!-- Reviewable:end --> --------- Co-authored-by: Claude <[email protected]>
1 parent 9404439 commit 8d2f1dd

File tree

7 files changed

+371
-14
lines changed

7 files changed

+371
-14
lines changed

.github/README.md

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# GitHub Actions CI/CD Configuration
2+
3+
This directory contains GitHub Actions workflows for continuous integration and deployment.
4+
5+
## PR Comment Commands
6+
7+
### `/run-skipped-ci` - Run Full CI Suite
8+
9+
When you open a PR, CI automatically runs a subset of tests for faster feedback (latest Ruby/Node versions only). To run the **complete CI suite** including all dependency combinations, add a comment to your PR:
10+
11+
```
12+
/run-skipped-ci
13+
```
14+
15+
This command will trigger:
16+
17+
- ✅ Main test suite with both latest and minimum supported versions
18+
- ✅ All example app generator tests
19+
- ✅ React on Rails Pro integration tests
20+
- ✅ React on Rails Pro package tests
21+
22+
The bot will:
23+
24+
1. React with a 🚀 to your comment
25+
2. Post a confirmation message with links to the triggered workflows
26+
3. Start all CI jobs on your PR branch
27+
28+
### Why This Exists
29+
30+
By default, PRs run a subset of CI jobs to provide fast feedback:
31+
32+
- Only latest dependency versions (Ruby 3.4, Node 22)
33+
- Skips example generator tests
34+
- Skips some Pro package tests
35+
36+
This is intentional to keep PR feedback loops fast. However, before merging, you should verify compatibility across all supported versions. The `/run-skipped-ci` command makes this easy without waiting for the PR to be merged to master.
37+
38+
### Security & Access Control
39+
40+
**Only repository collaborators with write access can trigger full CI runs.** This prevents:
41+
42+
- Resource abuse from external contributors
43+
- Unauthorized access to Pro package tests
44+
- Potential DoS attacks via repeated CI runs
45+
46+
If an unauthorized user attempts to use `/run-skipped-ci`, they'll receive a message explaining the restriction.
47+
48+
### Concurrency Protection
49+
50+
Multiple `/run-skipped-ci` comments on the same PR will cancel in-progress runs to prevent resource waste and duplicate results.
51+
52+
## Testing Comment-Triggered Workflows
53+
54+
**Important**: Comment-triggered workflows (`issue_comment` event) only execute from the **default branch** (master). This creates a chicken-and-egg problem when developing workflow changes.
55+
56+
### Recommended Testing Approach
57+
58+
1. **Develop the workflow**: Create/modify the workflow in your feature branch
59+
2. **Test locally**: Validate YAML syntax and logic as much as possible
60+
3. **Merge to master**: The workflow must be in master to be triggered by comments
61+
4. **Test on a PR**: Create a test PR and use the comment command to verify
62+
63+
### Why This Limitation Exists
64+
65+
GitHub Actions workflows triggered by `issue_comment` events always use the workflow definition from the default branch, not the PR branch. This is a security feature to prevent malicious actors from modifying workflows through PRs.
66+
67+
For more details, see [GitHub's documentation on issue_comment events](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#issue_comment).
68+
69+
## Available Workflows
70+
71+
### CI Workflows (Triggered on Push/PR)
72+
73+
- **`main.yml`** - Main test suite (dummy app integration tests)
74+
- **`lint-js-and-ruby.yml`** - Linting for JavaScript and Ruby code
75+
- **`package-js-tests.yml`** - JavaScript unit tests for the package
76+
- **`rspec-package-specs.yml`** - RSpec tests for the Ruby package
77+
- **`examples.yml`** - Generator tests for example apps
78+
- **`playwright.yml`** - Playwright E2E tests
79+
- **`pro-integration-tests.yml`** - Pro package integration tests
80+
- **`pro-package-tests.yml`** - Pro package unit tests
81+
- **`pro-lint.yml`** - Pro package linting
82+
83+
### Utility Workflows
84+
85+
- **`run-skipped-ci.yml`** - Triggered by `/run-skipped-ci` comment on PRs
86+
- **`pr-welcome-comment.yml`** - Auto-comments on new PRs with helpful info
87+
- **`detect-changes.yml`** - Detects which parts of the codebase changed
88+
89+
### Code Review Workflows
90+
91+
- **`claude.yml`** - Claude AI code review
92+
- **`claude-code-review.yml`** - Additional Claude code review checks
93+
94+
### Other Workflows
95+
96+
- **`check-markdown-links.yml`** - Validates markdown links
97+
98+
## Workflow Permissions
99+
100+
Most workflows use minimal permissions. The comment-triggered workflows require:
101+
102+
- `contents: read` - To read the repository code
103+
- `pull-requests: write` - To post comments and reactions
104+
- `actions: write` - To trigger other workflows
105+
106+
## Conditional Execution
107+
108+
Many workflows use change detection to skip unnecessary jobs:
109+
110+
- Runs all jobs on pushes to `master`
111+
- Runs only relevant jobs on PRs based on changed files
112+
- Can be overridden with `workflow_dispatch` or `/run-skipped-ci` command
113+
114+
See `script/ci-changes-detector` for the change detection logic.

.github/workflows/examples.yml

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,17 +38,23 @@ jobs:
3838

3939
examples:
4040
needs: detect-changes
41-
if: github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_generators == 'true'
41+
# Run on master, workflow_dispatch, OR when generators needed
42+
if: |
43+
github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_generators == 'true'
4244
strategy:
4345
fail-fast: false
4446
matrix:
4547
include:
4648
# Always run: Latest versions (fast feedback on PRs)
4749
- ruby-version: '3.4'
4850
dependency-level: 'latest'
49-
# Master only: Minimum supported versions (full coverage)
51+
# Master and workflow_dispatch: Minimum supported versions (full coverage)
5052
- ruby-version: '3.2'
5153
dependency-level: 'minimum'
54+
exclude:
55+
# Skip minimum dependency matrix on regular PRs (run only on master/workflow_dispatch)
56+
- ruby-version: ${{ github.event_name == 'pull_request' && github.ref != 'refs/heads/master' && '3.2' || '' }}
57+
dependency-level: ${{ github.event_name == 'pull_request' && github.ref != 'refs/heads/master' && 'minimum' || '' }}
5258
env:
5359
SKIP_YARN_COREPACK_CHECK: 0
5460
BUNDLE_FROZEN: ${{ matrix.dependency-level == 'minimum' && 'false' || 'true' }}

.github/workflows/main.yml

Lines changed: 16 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -38,20 +38,25 @@ jobs:
3838

3939
build-dummy-app-webpack-test-bundles:
4040
needs: detect-changes
41-
# Run on master OR when tests needed on PR (but skip minimum deps on PR)
41+
# Run on master, workflow_dispatch, OR when tests needed on PR
4242
if: |
43-
(github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_dummy_tests == 'true')
43+
github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_dummy_tests == 'true'
4444
strategy:
4545
matrix:
4646
include:
4747
# Always run: Latest versions (fast feedback on PRs)
4848
- ruby-version: '3.4'
4949
node-version: '22'
5050
dependency-level: 'latest'
51-
# Master only: Minimum supported versions (full coverage)
51+
# Master and workflow_dispatch: Minimum supported versions (full coverage)
5252
- ruby-version: '3.2'
5353
node-version: '20'
5454
dependency-level: 'minimum'
55+
exclude:
56+
# Skip minimum dependency matrix on regular PRs (run only on master/workflow_dispatch)
57+
- ruby-version: ${{ github.event_name == 'pull_request' && github.ref != 'refs/heads/master' && '3.2' || '' }}
58+
node-version: ${{ github.event_name == 'pull_request' && github.ref != 'refs/heads/master' && '20' || '' }}
59+
dependency-level: ${{ github.event_name == 'pull_request' && github.ref != 'refs/heads/master' && 'minimum' || '' }}
5560
runs-on: ubuntu-22.04
5661
steps:
5762
- uses: actions/checkout@v4
@@ -122,9 +127,9 @@ jobs:
122127

123128
dummy-app-integration-tests:
124129
needs: [detect-changes, build-dummy-app-webpack-test-bundles]
125-
# Run on master OR when tests needed on PR (but skip minimum deps on PR)
130+
# Run on master, workflow_dispatch, OR when tests needed on PR
126131
if: |
127-
(github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_dummy_tests == 'true')
132+
github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_dummy_tests == 'true'
128133
strategy:
129134
fail-fast: false
130135
matrix:
@@ -133,10 +138,15 @@ jobs:
133138
- ruby-version: '3.4'
134139
node-version: '22'
135140
dependency-level: 'latest'
136-
# Master only: Minimum supported versions (full coverage)
141+
# Master and workflow_dispatch: Minimum supported versions (full coverage)
137142
- ruby-version: '3.2'
138143
node-version: '20'
139144
dependency-level: 'minimum'
145+
exclude:
146+
# Skip minimum dependency matrix on regular PRs (run only on master/workflow_dispatch)
147+
- ruby-version: ${{ github.event_name == 'pull_request' && github.ref != 'refs/heads/master' && '3.2' || '' }}
148+
node-version: ${{ github.event_name == 'pull_request' && github.ref != 'refs/heads/master' && '20' || '' }}
149+
dependency-level: ${{ github.event_name == 'pull_request' && github.ref != 'refs/heads/master' && 'minimum' || '' }}
140150
runs-on: ubuntu-22.04
141151
steps:
142152
- uses: actions/checkout@v4
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
name: PR Welcome Comment
2+
3+
on:
4+
pull_request:
5+
types: [opened]
6+
7+
jobs:
8+
welcome:
9+
# Skip for bots (dependabot, renovate, etc.)
10+
if: github.event.pull_request.user.type != 'Bot'
11+
runs-on: ubuntu-22.04
12+
permissions:
13+
pull-requests: write
14+
steps:
15+
- name: Add welcome comment with CI command info
16+
uses: peter-evans/create-or-update-comment@v4
17+
with:
18+
issue-number: ${{ github.event.pull_request.number }}
19+
body: |
20+
👋 Thanks for opening this PR!
21+
22+
### 🚀 Running Full CI Suite
23+
24+
By default, PRs run a subset of CI jobs for faster feedback (latest Ruby/Node versions only).
25+
26+
To run the **complete CI suite** including all dependency combinations and skipped jobs, comment:
27+
28+
```
29+
/run-skipped-ci
30+
```
31+
32+
This will trigger:
33+
- ✅ Minimum supported versions (Ruby 3.2, Node 20)
34+
- ✅ All example app tests
35+
- ✅ Pro package integration tests
36+
- ✅ All test matrices
37+
38+
The full CI suite takes longer but ensures compatibility across all supported versions before merging.

.github/workflows/pro-integration-tests.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ jobs:
3434
# Build webpack test bundles for dummy app
3535
build-dummy-app-webpack-test-bundles:
3636
needs: detect-changes
37-
if: github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_pro_tests == 'true'
37+
if: github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_pro_tests == 'true'
3838
runs-on: ubuntu-22.04
3939
env:
4040
REACT_ON_RAILS_PRO_LICENSE: ${{ secrets.REACT_ON_RAILS_PRO_LICENSE_V2 }}
@@ -124,7 +124,7 @@ jobs:
124124
needs:
125125
- detect-changes
126126
- build-dummy-app-webpack-test-bundles
127-
if: github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_pro_tests == 'true'
127+
if: github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_pro_tests == 'true'
128128
runs-on: ubuntu-22.04
129129
env:
130130
REACT_ON_RAILS_PRO_LICENSE: ${{ secrets.REACT_ON_RAILS_PRO_LICENSE_V2 }}
@@ -304,7 +304,7 @@ jobs:
304304
needs:
305305
- detect-changes
306306
- build-dummy-app-webpack-test-bundles
307-
if: github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_pro_tests == 'true'
307+
if: github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_pro_tests == 'true'
308308
runs-on: ubuntu-22.04
309309
env:
310310
REACT_ON_RAILS_PRO_LICENSE: ${{ secrets.REACT_ON_RAILS_PRO_LICENSE_V2 }}

.github/workflows/pro-package-tests.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ jobs:
3434
# Build webpack test bundles for dummy app
3535
build-dummy-app-webpack-test-bundles:
3636
needs: detect-changes
37-
if: github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_pro_tests == 'true'
37+
if: github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_pro_tests == 'true'
3838
runs-on: ubuntu-22.04
3939
env:
4040
REACT_ON_RAILS_PRO_LICENSE: ${{ secrets.REACT_ON_RAILS_PRO_LICENSE_V2 }}
@@ -124,7 +124,7 @@ jobs:
124124
needs:
125125
- detect-changes
126126
- build-dummy-app-webpack-test-bundles
127-
if: github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_pro_tests == 'true'
127+
if: github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_pro_tests == 'true'
128128
runs-on: ubuntu-22.04
129129
# Redis service container
130130
services:
@@ -205,7 +205,7 @@ jobs:
205205
# RSpec tests for Pro package
206206
rspec-package-specs:
207207
needs: detect-changes
208-
if: github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_pro_tests == 'true'
208+
if: github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_pro_tests == 'true'
209209
strategy:
210210
matrix:
211211
ruby-version: ['3.3.7']

0 commit comments

Comments
 (0)