Add process-isolated philosopher executor with hard timeout mode - #470
Add process-isolated philosopher executor with hard timeout mode#470hiroshitanaka-creator wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5e3d0a9849
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if elapsed_ms > timeout_s * 1000 and outcome.error is None: | ||
| # A thread future can time out while the underlying work keeps running, | ||
| # so thread mode cannot guarantee production-safe cancellation. | ||
| return ExecOutcome([], 0, True, _soft_timeout_error(timeout_s, "thread"), elapsed_ms, pid) |
There was a problem hiding this comment.
Preserve timed_out for cooperative budget expiry
When a thread-mode philosopher polls ExecutionBudget and stops cooperatively after the deadline, run_one_philosopher() returns an ExecOutcome with error="Execution stopped cooperatively …". This branch now converts only the error is None case into a timeout, so budget-aware philosophers are reported as generic failures (timed_out=False) instead of timeouts. That breaks the timeout contract for cooperative cancellation and regresses tests/unit/test_party_machine_cancellation_budget.py::test_budget_aware_philosopher_stops_promptly_when_cancelled.
Useful? React with 👍 / 👎.
| settings = Settings.from_env() | ||
| executor = build_executor( | ||
| ExecutorConfig( | ||
| mode=settings.philosopher_execution_mode, |
There was a problem hiding this comment.
Use the caller's configured execution mode
run_turn() and app.api.run(settings=...) already pass a populated Settings object through the pipeline, but this code ignores it and re-reads Settings.from_env() instead. As a result, Settings(philosopher_execution_mode="process") and create_app(settings=APISettings(...)) still select the thread executor unless PO_PHILOSOPHER_EXECUTION_MODE is also exported, so the new toggle is effectively a no-op for programmatic configuration. The async fallback path repeats the same mistake in AsyncPartyMachine._dispatch_one().
Useful? React with 👍 / 👎.
| async def run_in_process_async(job: SerializedJob) -> ExecOutcome: | ||
| return await asyncio.to_thread(_run_one_in_subprocess, job) |
There was a problem hiding this comment.
Respect max_workers in async process mode
In process mode, each _dispatch_one() task awaits run_in_process_async(), and this helper uses asyncio.to_thread() rather than the machine's bounded executor. Because AsyncPartyMachine.run() creates one task per philosopher, max_workers no longer limits concurrency here: even with max_workers=1, multiple synchronous philosophers can launch subprocesses at once, which can oversubscribe the host and invalidate the async API's latency/resource budgeting.
Useful? React with 👍 / 👎.
Motivation
processmode.Description
PO_PHILOSOPHER_EXECUTION_MODE(supported values:thread,process) and expose it viaSettings.from_env()and the APIAPISettingsmodel, defaulting tothreadfor backward compatibility.src/po_core/runtime/philosopher_executor.pyimplementing aPhilosopherExecutorabstraction withThreadPhilosopherExecutor(compat soft-timeout behavior) andProcessPhilosopherExecutor(process-isolated execution usingmultiprocessing.Process+ queue), including a deterministic indexing/ordering contract and authoritative timeout semantics (worker teardown and recycle on timeout).run_philosophers()and theAsyncPartyMachinesynchronous fallback through the new executor abstraction (build_executor/run_in_process_async) so behavior selection is centralized while keeping public APIs unchanged.processmode for production hard-stop safety..env.exampleto document the newPO_PHILOSOPHER_EXECUTION_MODEvariable and add tests covering the new behavior (tests/execution/test_process_executor_timeout.pyandtests/execution/test_process_executor_ordering.py).Testing
pytest tests/execution/test_process_executor_timeout.py -qand it passed (verifies blocking philosopher times out authoritatively and does not contribute proposals).pytest tests/execution/test_process_executor_ordering.py -qand it passed (verifies deterministic proposal ordering inprocessmode).pytest tests/runtime/test_settings_from_env.py -q,pytest tests/execution/test_timeout_contract.py -q, andpytest tests/execution/test_async_timeout_contract.py -qand they all passed to ensure backward-compatible timeout contracts and settings parsing remain stable.Codex Task