Skip to content

Commit 00e06ab

Browse files
fix(microvm): reuse runner configuration path
1 parent 7b9f2e7 commit 00e06ab

11 files changed

Lines changed: 98 additions & 162 deletions

File tree

lambdas/libs/compute-providers/aws/microvm/README.md

Lines changed: 12 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -7,27 +7,28 @@ The MicroVM image `/run` hook receives this `runHookPayload`:
77
```json
88
{
99
"version": 1,
10-
"runnerConfigSsmArn": "arn:aws:ssm:eu-west-1:123456789012:parameter/github-action-runners/example/runners/config",
10+
"runnerConfigSsmPath": "/github-action-runners/example/runners/config",
1111
"runnerTokenSsmPath": "/github-action-runners/example/runners/tokens"
1212
}
1313
```
1414

15-
Lambda adds `microvmId` beside that payload. The image must poll the SecureString parameter at `<runnerTokenSsmPath>/<microvmId>`, start the GitHub runner with its encoded JIT configuration, delete the parameter after reading it, and exit its lifecycle entrypoint after the job completes. `runnerConfigSsmArn` is the configuration ARN prefix; the image reads its own non-secret metadata at `<runnerConfigSsmArn>/microvm-metadata/<microvmId>`. Neither identifier contains the JIT configuration value. Trusted control-plane cleanup and the fixed lifetime remain termination backstops.
15+
Lambda adds `microvmId` beside that payload. The image must poll the SecureString parameter at `<runnerTokenSsmPath>/<microvmId>`, start the GitHub runner with its encoded JIT configuration, delete the parameter after reading it, and exit its lifecycle entrypoint after the job completes. The image reads its own non-secret metadata at `<runnerConfigSsmPath>/microvm-metadata/<microvmId>`. Neither path contains the JIT configuration value. Trusted control-plane cleanup and the fixed lifetime remain termination backstops.
1616

1717
Runner ownership and lifecycle state are stored separately as non-secret `String`
1818
parameters under `<MICROVM_METADATA_SSM_PATH>/<microvmId>`. The immutable base
1919
record and independent state parameters prevent concurrent GitHub ID, orphan,
2020
and cleanup updates from overwriting one another. Deleting the JIT SecureString
2121
does not delete this metadata. Use a dedicated metadata prefix that does not
22-
overlap the JIT path, and do not grant the MicroVM execution role access to it.
23-
The control plane retries pending cleanup, removes metadata after termination,
24-
and reconciles expired records during inventory.
22+
overlap the JIT path, and grant the MicroVM execution role only the exact
23+
value-read access described below, without path-listing permissions. The control
24+
plane retries pending cleanup, removes metadata after termination, and reconciles
25+
expired records during inventory.
2526

2627
The immutable base metadata parameter is also the canonical tag surface for a
27-
runner. It merges `SSM_PARAMETER_STORE_TAGS` with the Terraform-generated
28-
`MICROVM_METADATA_TAGS`. Terraform supplies `Name`, `ghr:environment`,
29-
`ghr:ssm_config_path`, and `ghr:runner_name_prefix`; the Lambda then adds
30-
authoritative runtime tags:
28+
runner. It starts with `SSM_PARAMETER_STORE_TAGS`, omits `Name`, and derives
29+
`ghr:environment`, `ghr:ssm_config_path`, and `ghr:runner_name_prefix` from
30+
the existing `ENVIRONMENT`, `SSM_CONFIG_PATH`, and `RUNNER_NAME_PREFIX`
31+
settings. The Lambda then adds authoritative runtime tags:
3132
`ghr:Application`, `ghr:created_by`, `ghr:environment`, `ghr:Owner`,
3233
`ghr:Type`, `ghr:microvm_id`, `ghr:microvm_image_arn`, and, when available,
3334
`ghr:microvm_image_version`. After JIT registration, the control plane adds
@@ -44,8 +45,6 @@ The control-plane Lambda requires these provider environment variables:
4445
- `MICROVM_INGRESS_NETWORK_CONNECTORS` (optional JSON array or comma-separated list)
4546
- `MICROVM_EGRESS_NETWORK_CONNECTORS` (optional JSON array or comma-separated list)
4647
- `MICROVM_METADATA_SSM_PATH` (dedicated SSM path for control-plane metadata)
47-
- `MICROVM_METADATA_TAGS` (optional JSON array of base tags for the canonical metadata parameter)
48-
- `MICROVM_RUNNER_CONFIG_SSM_ARN` (runner configuration SSM ARN prefix passed to the image hook)
4948
- `MICROVM_LOG_GROUP` (optional)
5049

5150
Each runner is launched with a fixed lifetime of 28,800 seconds (8 hours).
@@ -64,8 +63,8 @@ that action does not currently support resource-level permissions, enforce the
6463
connector boundary with the explicit dynamic-label allowlist described below.
6564

6665
All MicroVMs using one execution role, JIT prefix, and metadata prefix share a
67-
trust boundary. Grant that role only `ssm:GetParameter` on
68-
`<runnerConfigSsmArn>/microvm-metadata/*`, `ssm:GetParameter` and
66+
trust boundary. Grant that role only `ssm:GetParameter` on the Parameter Store
67+
ARN corresponding to `<runnerConfigSsmPath>/microvm-metadata/*`, `ssm:GetParameter` and
6968
`ssm:DeleteParameter` on the lane-scoped JIT prefix, and the runtime log
7069
permissions described above. The image must address its own metadata with its
7170
AWS-provided `microvmId` and must not receive path-listing access. IAM cannot

lambdas/libs/compute-providers/aws/microvm/src/control-plane/config.test.ts

Lines changed: 0 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,6 @@ beforeEach(() => {
99
process.env.MICROVM_IMAGE_ARN = 'arn:aws:lambda:eu-west-1:123456789012:microvm-image:runner';
1010
process.env.MICROVM_EXECUTION_ROLE_ARN = 'arn:aws:iam::123456789012:role/microvm-runner';
1111
process.env.MICROVM_METADATA_SSM_PATH = '/github-action-runners/unit-test/microvm-metadata/';
12-
process.env.MICROVM_RUNNER_CONFIG_SSM_ARN =
13-
'arn:aws:ssm:eu-west-1:123456789012:parameter/github-action-runners/unit-test/config';
14-
delete process.env.MICROVM_METADATA_TAGS;
1512
delete process.env.MICROVM_IMAGE_VERSION;
1613
delete process.env.MICROVM_INGRESS_NETWORK_CONNECTORS;
1714
delete process.env.MICROVM_EGRESS_NETWORK_CONNECTORS;
@@ -27,8 +24,6 @@ describe('loadMicrovmProviderConfig', () => {
2724
ingressNetworkConnectors: undefined,
2825
egressNetworkConnectors: undefined,
2926
metadataSsmPath: '/github-action-runners/unit-test/microvm-metadata',
30-
metadataTags: [],
31-
runnerConfigSsmArn: process.env.MICROVM_RUNNER_CONFIG_SSM_ARN,
3227
logging: undefined,
3328
});
3429
});
@@ -38,19 +33,11 @@ describe('loadMicrovmProviderConfig', () => {
3833
process.env.MICROVM_INGRESS_NETWORK_CONNECTORS = '["arn:ingress:one","arn:ingress:two"]';
3934
process.env.MICROVM_EGRESS_NETWORK_CONNECTORS = 'arn:egress:one, arn:egress:two';
4035
process.env.MICROVM_LOG_GROUP = ' /aws/lambda-microvms/runner ';
41-
process.env.MICROVM_METADATA_TAGS = JSON.stringify([
42-
{ Key: 'Name', Value: 'unit-test-runner' },
43-
{ Key: 'ghr:environment', Value: 'unit-test' },
44-
]);
4536

4637
expect(loadMicrovmProviderConfig()).toMatchObject({
4738
imageVersion: '3.0',
4839
ingressNetworkConnectors: ['arn:ingress:one', 'arn:ingress:two'],
4940
egressNetworkConnectors: ['arn:egress:one', 'arn:egress:two'],
50-
metadataTags: [
51-
{ Key: 'Name', Value: 'unit-test-runner' },
52-
{ Key: 'ghr:environment', Value: 'unit-test' },
53-
],
5441
logging: { cloudWatch: { logGroup: '/aws/lambda-microvms/runner' } },
5542
});
5643
});
@@ -59,7 +46,6 @@ describe('loadMicrovmProviderConfig', () => {
5946
['MICROVM_IMAGE_ARN', 'MICROVM_IMAGE_ARN'],
6047
['MICROVM_EXECUTION_ROLE_ARN', 'MICROVM_EXECUTION_ROLE_ARN'],
6148
['MICROVM_METADATA_SSM_PATH', 'MICROVM_METADATA_SSM_PATH'],
62-
['MICROVM_RUNNER_CONFIG_SSM_ARN', 'MICROVM_RUNNER_CONFIG_SSM_ARN'],
6349
])('requires %s', (environmentVariable, expectedName) => {
6450
delete process.env[environmentVariable];
6551

@@ -84,32 +70,4 @@ describe('loadMicrovmProviderConfig', () => {
8470
);
8571
},
8672
);
87-
88-
it.each([
89-
'/github-action-runners/unit-test/config',
90-
'arn:aws:ssm:eu-west-1:123456789012:parameter',
91-
'arn:aws:s3:eu-west-1:123456789012:parameter/github-action-runners/unit-test/config',
92-
'arn:custom:ssm:eu-west-1:123456789012:parameter/github-action-runners/unit-test/config',
93-
'arn:aws:ssm:eu-west-1:123456789012:parameter/github-action-runners//config',
94-
'arn:aws:ssm:eu-west-1:123456789012:parameter/github-action-runners/../config',
95-
])('rejects malformed runner configuration SSM ARN %s', (runnerConfigSsmArn) => {
96-
process.env.MICROVM_RUNNER_CONFIG_SSM_ARN = runnerConfigSsmArn;
97-
98-
expect(() => loadMicrovmProviderConfig()).toThrow(
99-
'MICROVM_RUNNER_CONFIG_SSM_ARN must be a valid SSM parameter ARN prefix',
100-
);
101-
});
102-
103-
it.each([
104-
'[not-json',
105-
'{}',
106-
'[{"Key":"Name"}]',
107-
'[{"Key":"","Value":"runner"}]',
108-
'[{"Key":"Name","Value":1}]',
109-
'[{"Key":"Name","Value":"one"},{"Key":"Name","Value":"two"}]',
110-
])('rejects malformed metadata tags %s', (tags) => {
111-
process.env.MICROVM_METADATA_TAGS = tags;
112-
113-
expect(() => loadMicrovmProviderConfig()).toThrow(/MICROVM_METADATA_TAGS must/);
114-
});
11573
});
Lines changed: 0 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,5 @@
11
import type { Logging, RunMicrovmCommandInput } from '@aws-sdk/client-lambda-microvms';
22

3-
export interface MicrovmMetadataTag {
4-
Key: string;
5-
Value: string;
6-
}
7-
83
export interface MicrovmProviderConfig {
94
egressNetworkConnectors?: string[];
105
executionRoleArn: string;
@@ -13,8 +8,6 @@ export interface MicrovmProviderConfig {
138
ingressNetworkConnectors?: string[];
149
logging?: Logging;
1510
metadataSsmPath: string;
16-
metadataTags: MicrovmMetadataTag[];
17-
runnerConfigSsmArn: string;
1811
}
1912

2013
function requiredEnvironmentValue(name: string, value: string | undefined): string {
@@ -38,18 +31,6 @@ function parseMetadataSsmPath(value: string | undefined): string {
3831
return path;
3932
}
4033

41-
function parseRunnerConfigSsmArn(value: string | undefined): string {
42-
const arn = requiredEnvironmentValue('MICROVM_RUNNER_CONFIG_SSM_ARN', value);
43-
if (
44-
!/^arn:aws(?:-[a-z0-9-]+)?:ssm:[A-Za-z0-9-]+:\d{12}:parameter\/[A-Za-z0-9_.\-/]+$/.test(arn) ||
45-
arn.includes('//') ||
46-
arn.split('/').includes('..')
47-
) {
48-
throw new Error('MICROVM_RUNNER_CONFIG_SSM_ARN must be a valid SSM parameter ARN prefix');
49-
}
50-
return arn;
51-
}
52-
5334
function parseNetworkConnectors(name: string, value: string | undefined): string[] | undefined {
5435
const configuredValue = optionalEnvironmentValue(value);
5536
if (!configuredValue) return undefined;
@@ -74,41 +55,6 @@ function parseNetworkConnectors(name: string, value: string | undefined): string
7455
return connectors.map((connector) => connector.trim());
7556
}
7657

77-
function parseMetadataTags(value: string | undefined): MicrovmMetadataTag[] {
78-
const configuredValue = optionalEnvironmentValue(value);
79-
if (!configuredValue) return [];
80-
81-
let tags: unknown;
82-
try {
83-
tags = JSON.parse(configuredValue);
84-
} catch (error) {
85-
throw new Error('MICROVM_METADATA_TAGS must be a JSON array of SSM tag objects', { cause: error });
86-
}
87-
88-
if (
89-
!Array.isArray(tags) ||
90-
tags.some(
91-
(tag) =>
92-
typeof tag !== 'object' ||
93-
tag === null ||
94-
!('Key' in tag) ||
95-
typeof tag.Key !== 'string' ||
96-
tag.Key.length === 0 ||
97-
!('Value' in tag) ||
98-
typeof tag.Value !== 'string',
99-
)
100-
) {
101-
throw new Error('MICROVM_METADATA_TAGS must be a JSON array of SSM tag objects');
102-
}
103-
104-
const typedTags = tags as MicrovmMetadataTag[];
105-
if (new Set(typedTags.map((tag) => tag.Key)).size !== typedTags.length) {
106-
throw new Error('MICROVM_METADATA_TAGS must not contain duplicate tag keys');
107-
}
108-
109-
return typedTags;
110-
}
111-
11258
export function loadMicrovmProviderConfig(): MicrovmProviderConfig {
11359
const logGroup = optionalEnvironmentValue(process.env.MICROVM_LOG_GROUP);
11460

@@ -125,8 +71,6 @@ export function loadMicrovmProviderConfig(): MicrovmProviderConfig {
12571
process.env.MICROVM_EGRESS_NETWORK_CONNECTORS,
12672
),
12773
metadataSsmPath: parseMetadataSsmPath(process.env.MICROVM_METADATA_SSM_PATH),
128-
metadataTags: parseMetadataTags(process.env.MICROVM_METADATA_TAGS),
129-
runnerConfigSsmArn: parseRunnerConfigSsmArn(process.env.MICROVM_RUNNER_CONFIG_SSM_ARN),
13074
logging: logGroup ? ({ cloudWatch: { logGroup } } satisfies RunMicrovmCommandInput['logging']) : undefined,
13175
};
13276
}

lambdas/libs/compute-providers/aws/microvm/src/control-plane/microvms.test.ts

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,6 @@ const config: MicrovmProviderConfig = {
4141
executionRoleArn: 'arn:aws:iam::123456789012:role/microvm-runner',
4242
egressNetworkConnectors: ['arn:egress'],
4343
metadataSsmPath,
44-
metadataTags: [{ Key: 'Name', Value: 'unit-test-runner' }],
45-
runnerConfigSsmArn: 'arn:aws:ssm:eu-west-1:123456789012:parameter/github-action-runners/unit-test/config',
4644
logging: { cloudWatch: { logGroup: '/aws/lambda-microvms/runner' } },
4745
};
4846
const ssmParameterStoreTags = [{ Key: 'CostCenter', Value: '1234' }];
@@ -111,23 +109,19 @@ describe('runMicrovmRunner', () => {
111109
source: 'scale-up-lambda',
112110
imageArn,
113111
imageVersion: '3.1',
114-
metadataTags: [{ Key: 'Name', Value: 'unit-test-runner' }],
115112
ssmParameterStoreTags,
116113
});
117114
});
118115

119116
it('rejects invalid metadata tags before launching a MicroVM', async () => {
120117
await expect(
121118
runMicrovmRunner({
122-
config: {
123-
...config,
124-
metadataTags: [{ Key: 'aws:microvm:image-arn', Value: imageArn }],
125-
},
119+
config,
126120
environment: 'unit-test',
127121
runHookPayload: '{}',
128122
runnerOwner: 'Codertocat',
129123
runnerType: 'Org',
130-
ssmParameterStoreTags: [],
124+
ssmParameterStoreTags: [{ Key: 'aws:microvm:image-arn', Value: imageArn }],
131125
source: 'scale-up-lambda',
132126
}),
133127
).rejects.toThrow('AWS-reserved tag prefix');

lambdas/libs/compute-providers/aws/microvm/src/control-plane/microvms.ts

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,6 @@ export async function runMicrovmRunner(input: RunMicrovmRunnerInput): Promise<st
8888
source: input.source,
8989
imageArn: input.config.imageIdentifier,
9090
imageVersion: input.config.imageVersion ?? 'version-validation',
91-
metadataTags: input.config.metadataTags,
9291
ssmParameterStoreTags: input.ssmParameterStoreTags,
9392
});
9493

@@ -127,7 +126,6 @@ export async function runMicrovmRunner(input: RunMicrovmRunnerInput): Promise<st
127126
source: input.source,
128127
imageArn,
129128
imageVersion,
130-
metadataTags: input.config.metadataTags,
131129
ssmParameterStoreTags: input.ssmParameterStoreTags,
132130
});
133131
} catch (error) {

lambdas/libs/compute-providers/aws/microvm/src/control-plane/runner-config.test.ts

Lines changed: 35 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,22 @@ vi.mock('./runner-metadata', async (importOriginal) => ({
2020

2121
const imageArn = 'arn:aws:lambda:eu-west-1:123456789012:microvm-image:runner';
2222
const metadataSsmPath = '/github-action-runners/unit-test/microvm-metadata';
23-
const runnerConfigSsmArn = 'arn:aws:ssm:eu-west-1:123456789012:parameter/github-action-runners/unit-test/config';
23+
const runnerConfigSsmPath = '/github-action-runners/unit-test/config';
2424
const githubClient = {} as Octokit;
2525
const createStartRunnerConfig = vi.fn<CreateStartRunnerConfig>();
26-
const ssmParameterStoreTags = [{ Key: 'CostCenter', Value: '1234' }];
26+
const ssmParameterStoreTags = [
27+
{ Key: 'CostCenter', Value: '1234' },
28+
{ Key: 'Name', Value: 'not-used-for-microvm-metadata' },
29+
{ Key: 'ghr:environment', Value: 'caller-cannot-override' },
30+
{ Key: 'ghr:runner_name_prefix', Value: 'caller-cannot-override' },
31+
{ Key: 'ghr:ssm_config_path', Value: 'caller-cannot-override' },
32+
];
33+
const microvmMetadataTags = [
34+
{ Key: 'CostCenter', Value: '1234' },
35+
{ Key: 'ghr:environment', Value: 'unit-test' },
36+
{ Key: 'ghr:runner_name_prefix', Value: 'unit-test-' },
37+
{ Key: 'ghr:ssm_config_path', Value: runnerConfigSsmPath },
38+
];
2739

2840
function runnerConfig(overrides: Partial<CreateGitHubRunnerConfig> = {}): CreateGitHubRunnerConfig {
2941
return {
@@ -49,8 +61,6 @@ beforeEach(() => {
4961
imageIdentifier: imageArn,
5062
executionRoleArn: 'arn:aws:iam::123456789012:role/microvm-runner',
5163
metadataSsmPath,
52-
metadataTags: [{ Key: 'Name', Value: 'unit-test-runner' }],
53-
runnerConfigSsmArn,
5464
});
5565
vi.mocked(runMicrovmRunner).mockResolvedValue('mvm-1');
5666
vi.mocked(setMicrovmGithubRunnerMetadata).mockResolvedValue();
@@ -60,17 +70,17 @@ beforeEach(() => {
6070
});
6171

6272
describe('createMicrovmRunHookPayload', () => {
63-
it('contains the versioned runner token path and configuration ARN', () => {
73+
it('contains the versioned runner token and configuration paths', () => {
6474
expect(
6575
JSON.parse(
6676
createMicrovmRunHookPayload({
67-
runnerConfigSsmArn,
77+
runnerConfigSsmPath,
6878
runnerTokenSsmPath: '/runner/token',
6979
}),
7080
),
7181
).toEqual({
7282
version: 1,
73-
runnerConfigSsmArn,
83+
runnerConfigSsmPath,
7484
runnerTokenSsmPath: '/runner/token',
7585
});
7686
});
@@ -102,13 +112,25 @@ describe('createMicrovmRunners', () => {
102112
expect(runMicrovmRunner).not.toHaveBeenCalled();
103113
});
104114

115+
it('requires an SSM config path', async () => {
116+
await expect(
117+
createMicrovmRunners(
118+
runnerConfig({ ssmConfigPath: '' }),
119+
1,
120+
githubClient,
121+
createStartRunnerConfig,
122+
'scale-up-lambda',
123+
),
124+
).resolves.toEqual({ instances: [], retryableErrorCount: 0, nonRetryableErrorCount: 1 });
125+
126+
expect(runMicrovmRunner).not.toHaveBeenCalled();
127+
});
128+
105129
it('rejects a metadata path that overlaps the JIT token path', async () => {
106130
vi.mocked(loadMicrovmProviderConfig).mockReturnValue({
107131
imageIdentifier: imageArn,
108132
executionRoleArn: 'arn:aws:iam::123456789012:role/microvm-runner',
109133
metadataSsmPath: '/github-action-runners/unit-test/token/metadata',
110-
metadataTags: [],
111-
runnerConfigSsmArn,
112134
});
113135

114136
await expect(
@@ -145,12 +167,12 @@ describe('createMicrovmRunners', () => {
145167
config: expect.objectContaining({ imageIdentifier: imageArn }),
146168
environment: 'unit-test',
147169
runHookPayload: createMicrovmRunHookPayload({
148-
runnerConfigSsmArn,
170+
runnerConfigSsmPath,
149171
runnerTokenSsmPath: '/github-action-runners/unit-test/token',
150172
}),
151173
runnerOwner: 'Codertocat',
152174
runnerType: 'Org',
153-
ssmParameterStoreTags,
175+
ssmParameterStoreTags: microvmMetadataTags,
154176
source: 'pool-lambda',
155177
});
156178
expect(createStartRunnerConfig).toHaveBeenCalledTimes(2);
@@ -184,17 +206,15 @@ describe('createMicrovmRunners', () => {
184206
imageVersion: '3.0',
185207
executionRoleArn: 'arn:aws:iam::123456789012:role/microvm-runner',
186208
metadataSsmPath,
187-
metadataTags: [{ Key: 'Name', Value: 'unit-test-runner' }],
188-
runnerConfigSsmArn,
189209
},
190210
environment: 'unit-test',
191211
runHookPayload: createMicrovmRunHookPayload({
192-
runnerConfigSsmArn,
212+
runnerConfigSsmPath,
193213
runnerTokenSsmPath: '/github-action-runners/unit-test/token',
194214
}),
195215
runnerOwner: 'Codertocat',
196216
runnerType: 'Org',
197-
ssmParameterStoreTags,
217+
ssmParameterStoreTags: microvmMetadataTags,
198218
source: 'scale-up-lambda',
199219
});
200220
expect(setMicrovmGithubRunnerMetadata).toHaveBeenCalledWith(metadataSsmPath, 'mvm-1', {

0 commit comments

Comments
 (0)