Skip to content

Commit ceb0d0d

Browse files
vegardxvegardx
authored
docs: add rate limits and batch size tuning guide (#5137)
Adds `docs/rate-limits-and-tuning.md` documenting: - GitHub API primary rate limits (`core` 5,000/hr, `actions_runner_registration` 10,000/hr) - Secondary rate limits (900 points/min per endpoint, 100 concurrent requests) - Point cost table (GET=1, POST=5) with source link - Maximum runner creation rate per App (~166–180/min) with derivation - AWS SSM Parameter Store 40 TPS shared limit and higher-throughput mode - EC2 CreateFleet undocumented rate limit - Service Quotas to raise (vCPU defaults are 5) - `batch_size` tradeoffs with concrete examples - Monitoring recommendations Also adds a cross-reference from `docs/configuration.md` where rate limits are mentioned but never explained. All numbers verified against GitHub's official docs source (`github/docs` repo) and AWS documentation.
1 parent 69e06b1 commit ceb0d0d

2 files changed

Lines changed: 161 additions & 1 deletion

File tree

‎docs/configuration.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,7 @@ The example for [ephemeral runners](examples/ephemeral.md) is based on the [defa
161161

162162
You can enable the job retry function to retry a job after a delay for a configured number of times. The function is disabled by default. To enable the function set `job_retry.enable` to `true`. The function will check the job status after a delay, and when the is still queued, it will create a new runner. The new runner is created in the same way as the others via the scale-up function. Hence the same configuration applies.
163163

164-
For checking the job status a API call is made to GitHub. Which can exhaust the GitHub API more quickly for larger deployments and cause rate limits. For larger deployment with a lot of frequent jobs having a small pool available could be a better choice.
164+
For checking the job status a API call is made to GitHub. Which can exhaust the GitHub API more quickly for larger deployments and cause rate limits. For larger deployment with a lot of frequent jobs having a small pool available could be a better choice. See [Rate Limits and Batch Size Tuning](rate-limits-and-tuning.md) for details on GitHub and AWS rate limits.
165165

166166
The option `job_retry.delay_in_seconds` is the delay before the job status is checked. The delay is increased by the factor `job_retry.delay_backoff` for each attempt. The upper bound for a delay is 900 seconds, which is the max message delay on SQS. The maximum number of attempts is configured via `job_retry.max_attempts`. The delay should be set to a higher value than the time it takes to start a runner.
167167

‎docs/rate-limits-and-tuning.md‎

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
# Rate Limits and Batch Size Tuning
2+
3+
Rate limits from both GitHub and AWS constrain how fast this module can scale runners. This guide documents the relevant limits, how `batch_size` interacts with them, and which AWS quotas to raise for larger deployments.
4+
5+
## GitHub API Rate Limits
6+
7+
### Primary rate limits
8+
9+
| Bucket | Limit | Scaling | Used by |
10+
|---|---|---|---|
11+
| `core` | 5,000 req/hour (base) | +50/user over 20, +50/repo over 20, max 12,500 | Token minting, `isJobQueued`, `listSelfHostedRunners` |
12+
| `actions_runner_registration` | 10,000 req/hour | Fixed | JIT config generation |
13+
14+
GHEC orgs may have a higher `core` base (10,000+).
15+
16+
### Secondary rate limits
17+
18+
These apply in addition to primary limits, regardless of authentication method:
19+
20+
| Constraint | Limit |
21+
|---|---|
22+
| Concurrent requests | **100** (shared across all REST + GraphQL endpoints) |
23+
| Points per endpoint per minute | **900** (REST), **2,000** (GraphQL) |
24+
| CPU time | 90s CPU per 60s real time |
25+
| Content creation | 80 requests/min, 500/hour |
26+
27+
#### Point costs
28+
29+
[Source: GitHub docs](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api#calculating-points-for-the-secondary-rate-limit)
30+
31+
| Request type | Points |
32+
|---|---|
33+
| `GET`, `HEAD`, `OPTIONS` | 1 |
34+
| `POST`, `PATCH`, `PUT`, `DELETE` | 5 |
35+
36+
Some REST API endpoints have a different point cost that is not shared publicly.
37+
38+
### Maximum runner creation rate per App
39+
40+
With an installation token cache (token mints ≈ 0 per runner), the per-runner API cost is:
41+
42+
| API call | Method | Points | Bucket |
43+
|---|---|---|---|
44+
| `isJobQueued` | GET | 1 | `core` |
45+
| `generateRunnerJitconfigForOrg` | POST | 5 | `actions_runner_registration` |
46+
47+
**Bottleneck: JIT config generation** — 900 points ÷ 5 points/call = **180 runners/minute** (burst).
48+
49+
Sustained: 10,000/hour ÷ 60 = **~166 runners/minute**.
50+
51+
Without a token cache, each runner also costs a `POST /app/installations/{id}/access_tokens` (5 points) against the `core` endpoint. This doesn't directly reduce JIT throughput (different endpoint) but competes with `isJobQueued` for the `core` hourly budget.
52+
53+
### GHES
54+
55+
Rate limits are **disabled by default** on GitHub Enterprise Server and must be explicitly enabled by the site admin. When enabled, the same formula applies.
56+
57+
### API calls per scale-up invocation
58+
59+
| API call | Bucket | Frequency |
60+
|---|---|---|
61+
| `POST /app/installations/{id}/access_tokens` | `core` | 1 per unique installation in batch (0 with token cache) |
62+
| `GET /actions/jobs/{id}` (isJobQueued) | `core` | 1 per message (if `enable_job_queued_check = true`) |
63+
| `POST /actions/runners/generate-jitconfig` | `actions_runner_registration` | 1 per instance created |
64+
| `GET /actions/runners` (listSelfHostedRunners) | `core` | Scale-down, pool |
65+
66+
## AWS Rate Limits
67+
68+
### SSM Parameter Store
69+
70+
| Metric | Default |
71+
|---|---|
72+
| Combined throughput (Get + Put) | **40 TPS** (shared per-account per-region) |
73+
74+
Each runner instance requires one `PutParameter` call for its JIT config. At 40 TPS shared across all operations in the account, a burst of 40+ concurrent writes will throttle.
75+
76+
**Higher throughput mode** raises the ceiling:
77+
78+
```bash
79+
aws ssm update-service-setting \
80+
--setting-id arn:aws:ssm:<region>:<account-id>:servicesetting/ssm/parameter-store/high-throughput-enabled \
81+
--setting-value true
82+
```
83+
84+
Cost: $0.05 per 10,000 API interactions beyond the standard tier.
85+
86+
### EC2 CreateFleet
87+
88+
The exact TPS limit for `CreateFleet` is not publicly documented. It uses a token-bucket algorithm per-account per-region. Empirically throttles at low single-digit TPS.
89+
90+
When throttled: HTTP 503, error code `RequestLimitExceeded`.
91+
92+
**To request an increase:** Open an AWS Support case (Support Center → Create case → Service limit increase → EC2). EC2 API rate limits are not available in the Service Quotas console.
93+
94+
## AWS Service Quotas
95+
96+
For deployments running more than a handful of concurrent runners:
97+
98+
| Quota | Default | How to raise |
99+
|---|---|---|
100+
| Running On-Demand Standard (A,C,D,H,I,M,R,T,Z) instances | **5 vCPUs** | Service Quotas console |
101+
| All Standard Spot Instance Requests | **5 vCPUs** | Service Quotas console |
102+
| EC2 CreateFleet API rate | Undocumented | AWS Support ticket |
103+
| SSM Parameter Store throughput | 40 TPS | `update-service-setting` (see above) |
104+
105+
**vCPU quotas are measured in vCPUs, not instance count.** Running 50× `c5.large` (2 vCPU each) requires a quota of at least 100 vCPUs.
106+
107+
## Tuning `batch_size`
108+
109+
`lambda_event_source_mapping_batch_size` controls how many SQS messages are delivered to a single Lambda invocation (default: 10).
110+
111+
### What batch_size affects
112+
113+
| Resource | batch_size=1 (100 jobs) | batch_size=10 (100 jobs) |
114+
|---|---|---|
115+
| Lambda invocations | 100 | 10 |
116+
| `CreateFleet` calls | 100 | 10 |
117+
| Token mints (without cache) | 100 | 10 (deduped per installation within batch) |
118+
| Token mints (with cache) | ~1 | ~1 |
119+
| `PutParameter` calls (JIT config) | 100 | 100 (same total) |
120+
| `isJobQueued` calls | 100 | 100 (same total) |
121+
| JIT config generation calls | 100 | 100 (same total) |
122+
123+
Larger `batch_size` reduces CreateFleet calls (the most constrained AWS API) and Lambda invocations. Per-runner work (SSM writes, JIT config, isJobQueued) stays the same total. SSM peak TPS is lower with larger batches because writes are serialized within each Lambda rather than concurrent across many.
124+
125+
### Tradeoffs
126+
127+
| batch_size | Pros | Cons |
128+
|---|---|---|
129+
| 1 | Simple, failures affect only one job | Most CreateFleet calls, highest EC2 API pressure |
130+
| 5–10 | Fewer CreateFleet calls, lower peak TPS on EC2/SSM | Longer Lambda execution, partial failures affect more jobs |
131+
132+
### `maximum_batching_window_in_seconds`
133+
134+
When `batch_size > 1`, Lambda waits up to this many seconds to fill the batch before invoking.
135+
136+
| Setting | Behavior |
137+
|---|---|
138+
| 0 (default) | Invoke immediately with available messages |
139+
| 5–10s | Accumulate messages, fewer invocations, better batching |
140+
141+
Higher windows improve batching efficiency but add latency to job pickup.
142+
143+
### Recommendations
144+
145+
| Deployment size | batch_size | Lambda timeout | Other actions |
146+
|---|---|---|---|
147+
| Small (<50 concurrent jobs) | 1–5 | 90s | Defaults work |
148+
| Medium (50–200) | 5–10 | 180s | Monitor SSM throttling |
149+
| Large (200+) | 10 | 300s | Enable SSM higher throughput, raise vCPU quotas, request CreateFleet rate increase |
150+
151+
## Monitoring
152+
153+
| What to watch | Source | Alert threshold |
154+
|---|---|---|
155+
| SSM `ThrottlingException` | CloudWatch Logs (scale-up Lambda) | Any sustained occurrence |
156+
| `GitHubAppRateLimitRemaining` | Custom metric (`metrics.enable = true`) | < 1000 remaining |
157+
| Lambda duration | scale-up Lambda CloudWatch metrics | > 80% of configured timeout |
158+
| `ApproximateAgeOfOldestMessage` | SQS build queue | > 60s |
159+
| DLQ message count | Dead letter queue | > 0 |
160+
| EC2 `RequestLimitExceeded` | CloudTrail | Any occurrence |

0 commit comments

Comments
 (0)