Repository navigation
fix(bun): report the real runtime and enforce worker memory budgets on Windows - #1730
Conversation
…n Windows Under Bun, REA recorded Node/V8 identity it did not observe and its JavaScript helper processes ran without a memory bound, because Bun ignores the V8 heap flags REA passes. Source-map trace Evidence claimed runtime node-v8 with Bun's synthetic V8 heap limit (the decoder then rejected that limit, so trace_web_source_location always failed), analysis resource failures reported heap limits Bun does not enforce, the HAR decoder claimed a fixed 192 MiB heap, and doctor reported Bun's Node.js compatibility version as the Node version. Identify the runtime once from process.versions. Source-map reports record id bun with the Bun and Node.js compatibility versions and a null v8_heap_limit_bytes; the decoder checks the heap budget only for V8 reports and rejects a report from a runtime other than the launching one. The analysis worker reports a null heap limit under Bun, HAR limitations and failures omit heap sizes under Bun, and the doctor node check reports Bun and passes when its Node.js compatibility version is supported. On Windows under Bun, each helper's owned Job Object now carries a commit limit of its heap budget plus a measured 512 MiB Bun runtime reserve (704 MiB for the codec and HAR decoder, 1536 MiB for the default analyzer). The native addon accepts an optional page-aligned job memory limit on process_spawn, verifies it by readback, associates a private completion port before the child is created, and reports a sticky memoryLimitReached and peak job commit on each poll; inspect reports jobMemoryLimit. A requested limit with an artifact lacking the control refuses the launch, and non-Windows launchers refuse a limit. Reaching the limit is classified by the job notification rather than Bun's exit status: the analyzer fails the current source with a memory resource failure reporting worker_process_memory_limit_bytes, the codec returns a memory resource failure, and the HAR decoder returns its memory failure with process_memory_limit_bytes. Bun source-map Evidence records process_memory_limit_bytes, verified against the launcher. Node.js launches and outputs are unchanged.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 24801218e2
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Bun enjoyers unite! |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 169ca86b35
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // A decoder that survived a failed allocation may still reply; its | ||
| // reply is not trusted as complete once the limit was reached. | ||
| if (completed.memoryLimitReached === true) | ||
| throw captureMemoryFailure(input, CAPTURE_PROCESS_MEMORY_LIMIT_REASON); |
There was a problem hiding this comment.
Preserve the completed decoder snapshot in memory failures
When a limited HAR decoder catches an allocation failure and exits 0 (the scenario this branch explicitly handles), runOwnedCommand has already returned its stdout/stderr and exit-status snapshot, but this throws a fresh error without passing completed; captureMemoryFailure therefore emits no captured_output or observed exit metadata, unlike the nonzero-exit path. Carry the snapshot into the memory failure through the existing redaction path so these local diagnostics are not discarded.
AGENTS.md reference: AGENTS.md:L11-L11
Useful? React with 👍 / 👎.
Summary
When REA runs on Bun, it describes a Node.js/V8 runtime it did not observe, and its JavaScript helper processes run with no memory bound. Bun reports
process.versions.node(its Node.js compatibility version, e.g.26.3.0) and V8-shapedgetHeapStatistics()values, but it is not V8 and ignores V8 flags such as--max-old-space-size. REA relies on those flags to bound three helpers it starts onprocess.execPath: the JavaScript source analyzer (--max-old-space-size=<max_heap_mb>, default 1024), the source-map codec (192 MiB) and the HAR decoder (192 MiB old generation).This PR:
process.versions(only Bun setsprocess.versions.bun) and records it truthfully. Under Bun,trace_web_source_locationnow works instead of always failing, and Evidence reports unenforced V8 heap limits asnull(or omits them) instead of inventing numbers.rea doctorsays it is running on Bun.resource_constraintfailure Node's V8 out-of-memory path returns.Node.js launches and outputs are unchanged, apart from a new
detailsobject on the doctornodecheck.No issue is linked; this was found while running REA under Bun on Windows.
Problem and expected behavior
Smallest triggers, all under Bun 1.4.2 on Windows x64:
rea trace-web-source-location <manifest> 0 <map> <url> 1 0 --jsonalways failed. The codec reportedruntime: { id: "node-v8", version: process.version, v8_heap_limit_bytes: getHeapStatistics().heap_size_limit }. Bun's synthetic heap limit (~470 MB) is above the decoder's 256 MiB check, so the decoder returnedAnalysisOutputError: Codec reported a V8 heap limit exceeding its independent process budget.Even had it passed, the Evidence would claim a V8 runtime and an enforced heap limit that did not exist.worker_old_space_mbandworker_heap_limit_bytes(Bun'sheap_size_limit) intoreported_limitsand coverage limits. Bun enforces neither.inspect_web_network_capturelimitations said "HAR uses a 192 MiB Node old-generation heap", and HAR memory failures reportedold_generation_heap_mib: 192. Bun does not enforce that heap.rea doctor --jsonreported{"name":"node","detail":"26.3.0"}with no sign that the runtime was Bun.@vue/compiler-sfcbrowser build) committed up to 50,537 MiB and then failed with a genericProviderAdapterError. A 34,098-entry HAR, which exhausts the 192 MiB heap on Node.js, committed 1,026 MiB.Expected: REA records the runtime it actually ran on, reports an unenforced limit as unavailable rather than as a number, and bounds each helper by the memory budget it already declares wherever the platform allows it.
Change and scope
Runtime identity
src/domain/runtimeVersion.ts: addsJavaScriptRuntimeandjavaScriptRuntime(process.versions), the single place that decides between Node.js and Bun. Adapters passprocess.versions; the domain module does not readprocess.Source-map trace (
TraceSourceMap.ts,webSourceLocation.ts,SourceMapDecoder.ts,WebSourceLocationService.ts):runtimeis a discriminated union:{ id: "node-v8", version, v8_heap_limit_bytes: <int> }(unchanged on Node.js){ id: "bun", version, node_compatibility_version, v8_heap_limit_bytes: null, process_memory_limit_bytes: <int> | null }The decoder applies the 256 MiB check only to V8 reports. It rejects a report whose runtime differs from the runtime that launched the codec (the codec always runs on
process.execPath; without this check abunreport would bypass the heap check), and a Bun report whoseprocess_memory_limit_bytesdiffers from the limit its launcher enforced. The final limitation states the budget that actually applied.JavaScript analysis worker: the
readyhandshake'sheap_limit_bytesis nullable; the worker server sendsnullunder Bun and the parent rejects a handshake that contradicts its own runtime. Under Bun,worker_heap_limit_bytesisnullandworker_old_space_mbis not reported.javaScriptAnalysisCoverageLimitsalready drops non-numeric values, so coverage omits the heap limit.HAR decoder (
CaptureRelease.ts,CaptureFailures.ts,HistoricalCaptureDecoder.ts):HAR_CAPTURE_HEAP_ENFORCEDgates the heap sizes in limitations,reported_limitsand remediation.Doctor:
DoctorHost.nodeVersionis replaced byruntime: JavaScriptRuntime. Thenodecheck keeps its name, becauseDoctorScopeandscripts/verify-package-discovery.mjsselect it by name. It carries structureddetails:{ runtime: "node", version }, or{ runtime: "bun", version, node_compatibility_version }with detailBun 1.4.2 (Node.js 26.3.0 compatibility). Under Bun the check passes when the Node.js compatibility version satisfiesSUPPORTED_NODE_VERSION_RANGE, the same version setup already gates on (SetupHostusesprocess.versions.node); the Windows native addon already supports Bun as a Node-API host (fix(windows): load native controls in Node-API hosts not named node.exe #1531).Audit of
srcforprocess.version,process.versions,node-v8,getHeapStatistics,heap_size_limit,nodeVersionandmax-old-space-size:TraceSourceMap.tsruntime record;SourceMapDecoder.tsheap checkWebSourceLocationService.tslimitation "192 MiB old-generation heap and one V8 worker"JavaScriptAnalysisWorkerServer.tsreadyheap_limit_bytes;JavaScriptAnalysisWorker.ts#limits()CaptureFailures.ts,HistoricalCaptureDecoder.tsHAR heap sizesDoctor.ts/DoctorDiagnostics.tsnodeVersionJavaScriptAnalysisWorker.tsparent result budget (heap_size_limit - heapUsed), checkpoint admission;JavaScriptAnalysisWorkerServer.tsGC trigger at halfheap_size_limitparent_result_budget_bytesis a derived admission budget, not a runtime claim, so it is still reported.JsonInputFile.tsnative-parse choice and assembly admission (total_available_size)total_available_sizeis ~33.6 GB, so these refusals are practically unreachable there.--max-old-space-sizeetc. passed to the codec, HAR decoder, EVM worker and analysis workerprlimitcap and does not report its 256 MiB value.NativeModuleResolver.ts--js-flags=--max-old-space-size=128JadxSession.tsheap_limit_mib;WindowsNativeLoader.tsprocess.versions.napiJob Object memory limit (Windows, Bun only)
Native addon (
native/windows/src/process.cc,authority.hpp):process_spawnaccepts an optional 6th argument: the job memory limit in bytes. It must be a positive safe integer and a whole number of pages; anything else fails withERROR_INVALID_PARAMETERbefore any process is created. Existing 4- and 5-argument calls are unchanged.createJob(limit)setsJOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE | JOB_OBJECT_LIMIT_JOB_MEMORYandJobMemoryLimit, then reads the policy back and requires the exact flags and exact limit. The page-alignment rule exists so the read-back can be exact, because the kernel stores the limit in pages.CreateProcessW. Eachprocess_polldrains it without waiting and records a stickymemoryLimitReachedwhenJOB_OBJECT_MSG_JOB_MEMORY_LIMIT(10) arrives. The drain runs before the exit observation, so an exit caused by a failed allocation is never reported without the notification. A limited job's poll also returnspeakJobMemoryBytes(PeakJobMemoryUsed). An unlimited job returns neither field, so the poll shape for existing launches is unchanged. The spawn result echoesjobMemoryLimitBytes.process_closereleases the port.inspect()creates a 64 MiB-limited probe job with a port, then reportsjobMemoryLimit: true.TypeScript plumbing:
WindowsNativeLoader:jobMemoryLimit: z.literal(true).optional(); optional because artifacts built before this change omit it.WindowsOwnedProcess: the trailing options are one object{ protocolStdin, memoryLimitBytes, authority }; the only other caller (scripts/lib/windows-protocol-input.mjs) is migrated. A requested limit fails closed: an artifact withoutjobMemoryLimitrefuses the launch instead of starting an unlimited process. The spawn read-back must equal the request. A limited job's poll is parsed with a strict schema that requires both new fields, and an unlimited job's poll rejects them.memoryLimitReachedis set beforeexitis emitted.ProviderProcess:OwnedProviderProcessSpawnOptions.memoryLimitBytesis honored only by the Windows owned-job launcher; every other launcher throws instead of ignoring it.memoryLimitReachedis passed through toProviderProcessSnapshot, andrunOwnedCommandcarries it via the spawn options.Policy (
src/process/JavaScriptWorkerMemoryLimit.ts, the single place this is decided):javaScriptWorkerMemoryLimitBytes(heapBudgetMiB)returnsheapBudgetMiB × 1 MiB + BUN_WORKER_RUNTIME_RESERVE_BYTESonly whenplatform === "win32"and the runtime is Bun; otherwisenull. Every call site spreadsmemoryLimitBytesand adds the new reporting fields only when it is non-null, so Node.js launch arguments, the native call, the poll schema, Evidence and limitations are unchanged by construction.Reserve derivation (
BUN_WORKER_RUNTIME_RESERVE_BYTES = 512 MiB):bun -e 1peaks at 52 MiB of job commit, but REA's workers commit far more before doing real work, because JavaScriptCore commits parsed and compiled module code. Peak job commit on trivial inputs, 5–8 uncapped runs each, on a 32-logical-processor host:512 MiB covers the largest observed baseline with about 40% margin. Resulting limits: 704 MiB for the codec and HAR decoder, and
max_heap_mb+ 512 MiB for the analyzer (1536 MiB by default).Classification relies on the job notification, not on exit status, because Bun reports a reached limit in several ways: exit 3 (panic), exit 1 (
RangeError: Out of memory), or0xC0000409(observed for the HAR decoder).AnalysisResourceConstraintError(…, "memory", …)from every path (exit, broken pipe or truncated frame, worker-caught exception, post-loop check). Completed static facts are retained, and the next source starts a fresh worker.worker_heap_limit_bytes: nullplusworker_process_memory_limit_bytes, which is also a coverage limitAnalysisResourceConstraintError; no partial resultcodec_process_memory_limit_bytes; Evidenceruntime.process_memory_limit_bytescaptureMemoryFailurefrom both the failed-exit path and a success path whose job reached the limitprocess_memory_limit_bytes, plus a remediation sentenceNot included: memory enforcement for Bun on Linux and macOS (they keep
null/ no-limit reporting), EVM (alreadyprlimit-capped), Bun support in installers orrea setup/rea update, and changes to therea doctorremediation wording.Docs:
installation.md,web-source-location.md,javascript-artifact-reconstruction.md,web-network-captures.md,testing.md,native/windows/README.md, and the skill referencejavascript-applications.md.Contract and boundary impact
src/domain/runtimeVersion.ts(runtime identity) andsrc/process/JavaScriptWorkerMemoryLimit.ts(limit policy); the nativeprocess_spawn/process_pollboundary, thenWindowsOwnedProcess, thenspawnOwnedProviderProcess. Producers: codec process, analysis worker server, HAR adapter, doctor host.trace_web_source_locationoutput schema'snormalized_result.runtimechanges from{ id: string, version: string, v8_heap_limit_bytes: int }to the two-variant union above; Node.js output is byte-identical. Themax_heap_mb/--max-heap-mbdescription states that Bun ignores V8 heap flags and that Windows limits committed memory instead.rea doctor --json'snodecheck gainsdetails; on Node.js,detailis unchanged.trace_web_source_locationworks instead of always failing. Under Bun on Windows the helpers require the matching native artifact; an older artifact refuses the launch with "does not support Job Object memory limits" instead of running unlimited.nullor omitted instead of fabricated. Enforced Job Object limits are reported (worker_process_memory_limit_bytes,codec_process_memory_limit_bytes,process_memory_limit_bytes) and limitations state which budget applied.docs/public/product-catalog.json), package, or installation impact: ignored catalogs andskills/regenerate from the changed schema, description and skill reference; no committed generated files changed (docs:check:builtandmcp-catalog:check:builtpass).native/windows/buildis rebuilt by CI.Evidence and regression coverage
Tests added or updated:
src/domain/runtimeVersion.test.tsprocess.versions-shaped input.src/application/Doctor.test.tsnodecheckdetailsfor Node.js; Bun passes with a supported compatibility version (26.3.0) and fails with an unsupported one (25.1.0).Doctor.fixture.tsandGhidraDoctor.test.tsmove to theruntimehost field.src/process/JavaScriptWorkerMemoryLimit.test.tssrc/windows/WindowsOwnedProcess.test.ts[false, limit], verifies the read-back and setsmemoryLimitReachedbeforeexit; rejects a limited poll missing the fields and a mismatched applied limit.src/windows/WindowsNativeLoader.test.tsjobMemoryLimitadmitted astrue, absent on earlier artifacts,falserejected.src/process/ProviderProcessOutput.test.tsmemoryLimitReachedonly when the handle observed it.src/browser/history/CaptureFailures.test.tsreported_limitsonly on Node.js; a reached limit after a Bun panic (exit 3) or exception (exit 1) → memoryresource_constraintwithprocess_memory_limit_bytes; the same exits without the notification stayexecution_failure.src/application/WebSourceLocationService.test.tstests/boundary/process/providerProcess.test.tstests/boundary/javascript/sourceMapDecoder.test.tstests/boundary/javascript/javascriptSourceAnalysisWorker.test.tsProviderAdapterError) by the notification, using a real child.tests/acceptance/webSourceLocationCli.test.tstests/acceptance/tools/historicalCaptureResources.test.tsscripts/verify-windows-native.mjs+scripts/lib/windows-job-memory-limit.mjsscripts/lib/windows-job-memory-limit.mjsasserts:inspect().jobMemoryLimit === true; invalid limits (0, negative, fractional,2**53,NaN,Infinity, non-page-aligned4097, a string,null) are rejected with Win32 87; an unlimited poll has neither new field; a 128 MiB-limited child allocating externalBuffermemory (which V8 flags cannot bound) exits nonzero without completing, reportsmemoryLimitReached, reachesactiveProcesses === 0, and peaks within [limit − 64 MiB, limit + 32 MiB]; a control child under the same limit exits 0 without reaching it; theWindowsOwnedProcesswrapper reports the reached limit; a missing capability is refused. Measured peaks exceed the limit by a few MiB of kernel-charged commit (512→520, 256→262, 704→708 MiB), hence the tolerance.Base reproduction or other evidence:
On base under Bun, three tests fail and pass with this change (see Validation): the real-codec
sourceMapDecodertest (Codec reported a V8 heap limit exceeding its independent process budget.),webSourceLocationCli(exit 1), and the HAR exhaustion acceptance test (nothing bounded the decoder).Throwaway probes outside the repo (a PowerShell P/Invoke job probe and a Bun FFI job helper), Bun 1.4.2: with a 512 MiB job limit Bun panicked (exit 3, peak 520 MiB); with 256 MiB it threw a JavaScript out-of-memory exception (exit 1, peak 262 MiB). In both cases the completion port received
JOB_OBJECT_MSG_JOB_MEMORY_LIMIT(10), the runtime-independent signal this change relies on.Before the native control was built, this change's TypeScript was run with a launcher that put the child into a nested FFI Job Object with the requested limit:
AnalysisResourceConstraintError, static facts retained0xC0000409; memoryresource_constraintwithprocess_memory_limit_bytes: 738197504AnalysisResourceConstraintErrorUser-visible CLI/MCP output (Bun 1.4.2, Windows x64, built CLI with the CI-built native artifact):
bun scripts\rea.mjs doctor --json,nodecheck:{ "name": "node", "ok": true, "classification": "healthy", "detail": "Bun 1.4.2 (Node.js 26.3.0 compatibility)", "details": { "runtime": "bun", "version": "1.4.2", "node_compatibility_version": "26.3.0" } }bun scripts\rea.mjs trace-web-source-location <manifest> 0 <map> "https://app.test/app.js.map?v=1#context" 1 0 --json(inputs written fromtests/fixtures/webSourceLocation.ts) exits 0 with one match and:bun scripts\rea.mjs analyze-javascript-application <dir> --format jsonon a directory with the 1,732,994-byte@vue/compiler-sfcbrowser bundle (src/big.js) and a one-line module (src/small.js): exit 0 in about 5 seconds.src/big.jsreportsresource_constraint/memory, "reached its 1610612736-byte process memory limit while analyzing src/big.js during semantic", with"reported_limits":{"worker_heap_limit_bytes":null,"worker_process_memory_limit_bytes":1610612736,"worker_timeout_ms":300000}.src/small.jsand itssmallexport are still recovered by static analysis.--max-heap-mb 128on a two-file app whosemain.jshas 4000 imports reaches the parent result-budget path and exits 0 with partial coverage. Thetarget.jsfailure reports"reported_limits":{"worker_heap_limit_bytes":null,"worker_process_memory_limit_bytes":671088640,"worker_timeout_ms":300000,"semantic_result_budget_bytes":0}; coverage limits arejavascript_analysis_semantic_result_budget_bytes,javascript_analysis_worker_process_memory_limit_bytes,javascript_analysis_worker_timeout_msandjavascript_analysis_parent_result_budget_bytes, with no worker heap limit.Remaining proof gaps:
javascriptSourceAnalysisWorker.test.tscannot run under Vitest on Bun: every test in the file, including the three existing ones on base, fails with[module runner] "import.meta.resolve" is not supported. Its Bun-side behavior is covered by the analyzer CLI runs above; CI runs the file under Node.js.For evidence-bearing changes:
Validation performed
Local validation ran under Bun 1.4.2 on Windows x64 without Node.js installed. npm-based wrapper scripts cannot run here, so the underlying scripts were run with
bun --bun. The native artifact built by this commit's CI was installed in the ignorednative/windows/build; base runs used the artifact from the publishedrea-agents@6.4.0package, because base's loader rejects the newjobMemoryLimitinspection key.bun --bun run compile;artifacts:skills,artifacts:mcp-catalog,artifacts:product-catalog,artifacts:managed-evidence,artifacts:error-schemabun --bun run typecheckbun --bun run lintCaptureFailuresmapping (complexity 59→63),HistoricalCaptureDecoder#inspect(30→31),JavaScriptAnalysisWorker#analyze(47→48; file 710→756 lines),verify-windows-native.mjs(675→680 lines).oxfmt --checkon touched filesbun --bun run docs:check:built,bun --bun run mcp-catalog:check:builtbun --bun run test:local --onJavaScriptWorkerMemoryLimit,WindowsOwnedProcess,WindowsNativeLoader,ProviderProcessOutput,CaptureFailures,WebSourceLocationService,runtimeVersion,Doctor,GhidraDoctorGhidraDoctorWindows-authority tests that fail identically on base.bun scripts/verify-windows-native.mjs .(full native conformance, under Bun instead of Node.js)ok: true, no false control.processes.jobMemoryLimit: the 128 MiB-limited child exited 1 at a peak of 145,399,808 bytes with the limit reached; the control exited 0 at 56,586,240 bytes; invalid limits, unlimited poll shape, wrapper reporting and missing-capability refusal alltrue.bun --bun node_modules/vitest/vitest.mjs runonsrc/application src/domain src/browser/history src/javascript src/ghidra src/process src/windows tests/boundary/javascript tests/boundary/process, thewebSourceLocationMcp,javascriptAnalysisResources,outputSchemaSemanticConstraintsandcompactInputSchemaMcpMCP boundary tests,targetDiagnostics, and thewebSourceLocationCliandhistoricalCaptureResourcesacceptance tests, on this branch and on basesourceMapDecodertest,webSourceLocationCli, and the HAR exhaustion acceptance test. Failing only on this branch: the 4 new tests injavascriptSourceAnalysisWorker.test.ts, which fail like the file's existing tests because Vitest's module runner on Bun does not supportimport.meta.resolve. Every other failure is the same test on both and environment-specific under Bun on Windows (e.g. thatimport.meta.resolvelimitation, POSIX-mode process tests, Linux/macOS Hopper installers, Ghidra Windows authority).doctor --json,trace-web-source-location,analyze-javascript-application(twice)CI on this commit (fork push run, Node.js on GitHub-hosted runners):
Build bundled Windows x64 native controls(MinGW build and artifact verification): passed. Itsrea-windows-x64.nodeSHA-256 is5fa70a01…11aac, byte-identical to the artifact used in the local runs above.Windows capability and package lane(npm run verify:windows-nativeon windows-latest, including thejobMemoryLimitcontrols): passed.Real Apple artifact CLI/MCP E2E (native-regressions)failed in "Run native Apple and process boundary regressions", which also fails theCI requiredgate. The same step failed on upstreammainat this PR's base 966dc3a (morluto/rea run 38116233537), and upstream has since changed its macOS lanes (3f4ea32, "ci: reduce macOS runner contention"). On macOS this change only adds thememoryLimitReachedpass-through and the refusal of a requested limit, which the policy never requests outside Windows.Compatibility, safety, and release
detailsobject on the doctornodecheck. Consumers validatingtrace_web_source_locationagainst the old schema could not have received Bun output before (it always failed). Bun-on-Windows users need the matching package artifact; an older artifact refuses the helper launch instead of running it unbounded.native/windows/buildartifact.Review checklist
type(scope): outcome.