Add FastAPI testing-environment status API (maintainer-gated) - #13278
Add FastAPI testing-environment status API (maintainer-gated)#13278RayBB wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
The Claude review below overstates the issues and confused itself.
Claude review:
Blocking
-
openlibrary/plugins/openlibrary/status.py:614: Creating the state file insetup()removes the only thing gating this UI. Right now the absence of_testing-prs.jsonis what keepsshow_testingfalse, andtemplates/status.html:51hangs the entire deploy table, the "Add PR(s)" form, and the Deploy button off it. Once every server writes the file at boot, any maintainer loading/statuson production gets that UI, and_trigger_rebuild()is one click away wheneverconfig.jenkins_tokenis set. It also makes the documented 404 atopenlibrary/fastapi/status.py:59unreachable, so a client can't tell "not a testing environment" from "nothing pinned". Let's drop the startup write and gate on something explicit, a config flag orget_deployment_name()inopenlibrary/core/env.py. -
openlibrary/plugins/openlibrary/status.py:424:TESTING_STATE_FILEis CWD-relative, and nothing guarantees the ASGI process shares a working directory with the web.py process.compose.production.yamldoesn't bind-mount the repo intoweborfast_web(staging and the override file do, which is why this looks fine locally), so each container ends up with its own copy. Writes happen inweb, reads infast_web, and/status/testing.jsonanswers200 {"prs": []}forever while the page shows the real list. That's the drift this PR is meant to remove. State that two processes share should live in the DB or memcache rather than a per-container file. -
openlibrary/fastapi/status.py:58: A read endpoint shouldn't write. On a cache missget_testing_status()falls through to_get_drift_info(), which mutates the PRs in place and calls_save_testing_state(). So atesting.jsonrequest that loaded state before a maintainer's/status/deploycan finish its GitHub refresh afterward and write the stale copy back, silently undoing the deploy. The same path makes up to two sequential GitHub calls per PR at a 5s timeout each with no overall budget, on a plainGET. Could the endpoint take a cache-only path that skips both the refresh and the save? -
openlibrary/plugins/openlibrary/status.py:431:_ensure_testing_state_file()writes at plugin-load time in every process that loads the OL plugins, including standalone scripts and the pytest run. AnOSErrorthere propagates out ofsetup()and the process fails to boot. Previously that file was only written inside maintainer-authenticated POST handlers, so an unwritable CWD was harmless. Theexists()check also won't repair a zero-byte file, andwrite_texttruncates before writing, so a crash mid-write leavesjson.loads("")raising a 500 on both/statusandtesting.json.
Has this been through a deploy shaped like production, with web and fast_web in separate containers? Most of the above only shows up there.
|
I’m not sure that any of this review from Claude makes sense. All of this is gated behind off, so only maintainers can see it. The testing state file is something that already exists and we can see it working in the ASGI process. That’s not something that was added as part of this pull request. it be a file or live in a database? I don’t know. I’m building on top of Mac's pull request. the endpoint shouldn write I think writing to the thing when you get updated information is fine I think this is basically what we're already doing. And the last one, I mean, it basically seems fine. need to either have every user create this file or create it automatically in some way. This isn't something that almost anyone will be using locally, but if we want people to be able to test it and develop it, then we need the setup. And also, obviously, it has not been through a production deploy because it's a pull request that I just opened. So I don't know if, your AI can take into account how testing and production actually works. Maybe it would make some more useful feedback. |
|
@RayBB one question, with the addition of _ensure_testing_state_file, would the deploy UI now show up on the production status page for maintainers? |
|
That's a great question. I think we should probably as a check so that it doesn't. In fact this really only needs to run in local envs. I'll do that. Don't let it stop you from thinking on the ui though |
|
@lokesh fixed that issue |
3892690 to
f6a2b0d
Compare
Expose the /status deploy table data (testing environment) via a new FastAPI endpoint GET /status/testing.json, gated to maintainers/admins. - Extract get_testing_status() as a shared source of truth used by both the legacy /status page and the new API - Add User.is_maintainer() and require_maintainer/MaintainerDep, mirroring the existing librarian auth dependency - Auto-create _testing-prs.json at startup (if missing) so the page and API work from first boot without a manual file - Add tests for the helper, endpoint, and auth gating
The deploy UI must not appear on the production status page. Gate the _ensure_testing_state_file() bootstrap behind get_ol_env().LOCAL_DEV so the file is only auto-created for local development.
cdda7c4 to
e367a73
Compare
What does this PR do?
Exposes the data behind the
/statusdeploy table ("Testing Environment") as a JSON API, so the status page can later be rebuilt as an interactive component.New FastAPI endpoint
https://testing.openlibrary.org/_fast/status/testing.json
GET /status/testing.json(maintainer-gated) returns the testing environment state:{ "last_deploy_at": "2026-08-05T18:00:00+00:00", "has_pending": true, "prs": [{ "pr": 13269, "title": "...", "commit": "<full sha>", "active": true, "pending_active": null, "added_at": "...", "added_by": "openlibrary", "author": "...", "author_avatar": "...", "assignee": "...", "assignee_avatar": "...", "pull_latest_sha": "", "head_sha": "abc1234", "drift": 0, "merged": false, "is_new": true }] }openlibrary/fastapi/status.py(FastAPI land, not web.py), with room for more status endpoints later.require_maintainerauth dependency — same access level as the existing/statuspage (/usergroup/maintainersor/usergroup/admin).404if no testing state exists.How it works
get_testing_status()extracted inopenlibrary/plugins/openlibrary/status.pyas a single source of truth, now shared by the legacy/statuspage and the new endpoint so they can't drift apart.User.is_maintainer()added to the User model; the web.py_is_maintainer()delegates to it._testing-prs.jsonis auto-created (empty) at app startup if missing, so the page/API work from first boot without a manual file. It is gitignored and never overwritten.Test plan
require_maintainer, and the state-file bootstrap.401openlibrary(admin) →200with full payload incl. GitHub drift (drift: 4,merged: true)Notes
include_in_schemais gated to local dev (LOCAL_DEV), matching the internal router pattern.