Evals UI: session 404, non-clickable results link, undocumented port change, unclear evalset discovery

[kellyoconor]

Environment

• ADK version: 1.29.x (installed via pip)
• Platform: macOS
• Running: adk web --session_service_uri="sqlite:///./sessions.db" .

Issues found

1. Eval session 404 — user_id mismatch between eval runner and UI

The eval runner saves sessions using session_input.user_id from the evalset file (defaults to test_user_id if not set). The frontend UI hardcodes userId="user" when fetching session results:

GET /apps/app/users/user/sessions/___eval___session___<uuid> → 404

The session exists in the DB under a different user_id (e.g. eval_user). The UI never finds it.

Workaround: Set "user_id": "user" in all evalset files' session_input blocks.

Suggested fix: The UI should use the user_id returned in the eval result rather than hardcoding "user".

2. "View eval run result" tooltip is not clickable

Even after fixing the 404 (sessions return 200 OK), the "View eval run result" text appears only as a tooltip on the Fail/Pass badge — it's not a hyperlink or button. There's no way to navigate to the session trace from the eval results view.

3. Undocumented port change from 8501 to 8000

adk web now serves on port 8000 instead of 8501. This isn't called out in migration/upgrade notes, causing confusion when switching between versions.

4. Evalset file discovery is fragile and undocumented

It's not obvious that .evalset.json files must live directly inside the agent directory (e.g. app/) for adk web to discover them. Attempting to organize them into a subdirectory (e.g. app/eval_sets/) causes them to silently disappear from the UI. The LocalEvalSetsManager only scans the immediate agent directory. This should be documented, or subdirectory scanning should be supported.

Steps to reproduce

1. Create evalset files with "user_id": "eval_user" in session_input
2. Run adk web --session_service_uri="sqlite:///./sessions.db" .
3. Run an eval from the UI
4. Observe 404 in server logs for session fetch
5. Change user_id to "user" — 404 resolves to 200
6. Observe "View eval run result" tooltip is still not clickable

[kellyoconor]

Additional UI detail (screenshot description)

For anyone trying to reproduce or understand the UX gap — here's what the eval tab looks like:

Layout: Dark-themed panel, tab nav across top (Trace, State, Artifacts, Sessions, Eval). Breadcrumb shows < equipment_restart_golden. Blue "Run Evaluation" button top-left.

Eval case table: 5 rows (GD-001 through GD-005), each with checkbox and a red Fail pill button. All 5 cases failed.

The broken tooltip: The Fail pills are styled as interactive buttons — pill shape, red outline, hover state fires, tooltip renders saying "View eval run result." But clicking does nothing. The affordance promises drill-down to the failure trace but no behavior is attached.

This is particularly painful for an eval tool because the entire value of seeing a Fail result is being one click from understanding why — the trace, expected vs actual, which tool call went wrong. Without that click-through, you're stuck at "5/5 failed" with no path to diagnosis.

Other missing eval UX basics:

• No summary row (e.g. "0/5 passing")
• No last-run timestamp
• No diff against prior runs
• Tooltip copy is generic ("View eval run result") — "View trace" or "See why this failed" would be more actionable

[surajksharma07]

@kellyoconor,

Two things worth trying as a local workaround:

in local_eval_service.py:268, change the fallback from 'test_user_id' to 'user' to align the backend with the UI's hardcoded userId, and replace os.listdir() with os.walk() in LocalEvalSetsManager.list_eval_sets() to fix subdirectory discovery silently dropping files.

Could you give both of those a try and confirm they hold up on your setup before the PRs land?