fix(tests): re-emit events until reactive automation triggers

[devin-ai-integration]

Summary

integration-tests/test_automation_assessments.py::test_reactive_automation flaked on main with TimeoutError (failed run, commit 0926dad).

Root cause — a load race, not the subscriber. The test created the automation, slept a fixed 1s (inside create_or_replace_automation), then emitted a single burst of 5 events. But a newly-created automation isn't necessarily loaded into the trigger services the instant the API call returns — loading happens via a Postgres NOTIFY (listen_for_automation_changes) or the periodic reconcile_automations (~5s). A reactive trigger only counts events that arrive after it is loaded (find_interested_triggers returns nothing otherwise, and events are never replayed). Under CI load the load can lag past the 1s wait, so the burst is silently dropped, the automation never fires, and the listener times out at 60s.

The subscriber side is not the culprit: the /events/out websocket backfills matching events (wants_backfill defaults to True), so the triggered event would be delivered even if the subscription registered slightly late.

Fix

Keep emitting the triggering events until the automation actually fires, instead of relying on a single burst landing after the automation is loaded (still bounded by the existing 60s anyio.fail_after):

with anyio.fail_after(60):
    async with get_events_client() as events:
        while not listener.done():
            for i in range(5):
                await events.emit(Event(event="integration.example.event", ...))
            await asyncio.wait({listener}, timeout=2)

Also catches builtin TimeoutError (what anyio.fail_after raises on Python 3.10, the CI version) rather than asyncio.TimeoutError — on 3.10 asyncio.TimeoutError is not TimeoutError, so the old except never matched and the raw TimeoutError escaped (matching the CI traceback).

Validation

Reproduced the exact CI failure and confirmed the fix by injecting an 8s automation-load lag into the server:

• old (single burst): FAILED ... Reactive automation did not trigger in 60s (61.98s)
• new (re-emit): 1 passed in 3.46s

Without the injected lag: test_reactive_automation passes 20/20 (--flake-runs=20), and the full test_automation_assessments.py module passes (4 passed).

Scope kept to the failing reactive test.

Checklist

• If this pull request adds or changes functionality, it includes tests or explains why tests are not needed.
  • This is a test-only change fixing a flaky integration test.

Link to Devin session: https://app.devin.ai/sessions/16e3d30e206c4b3e9b80fca076a9d174

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7e6747367c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Await the listener result after it completes

If wait_for_event finishes with an exception, for example because the websocket disconnects before delivering the triggered event, listener.done() becomes true and this loop exits without ever awaiting or inspecting the task result, so the reactive automation test can pass despite never observing the automation event. Please await listener after the loop, or check the done result returned by asyncio.wait, so subscriber failures still fail the test.

Useful? React with 👍 / 👎.

[devin-ai-integration]

Good catch — fixed in cdd117e. Added await listener after the emit loop so a subscriber failure (e.g. websocket disconnect) surfaces its exception instead of the test passing just because the task finished.