dashboard/app: flaky TestAIJobParallelPoll

[a-nogikh]

The problem was hit again on #6975 :(
The PR was on top of our current master, so it contains the previous fix attempts.

https://github.com/google/syzkaller/actions/runs/23460506635/job/68260682567?pr=6975

[a-nogikh]

FTR asked gemini-cli to analyze the log:

  1. Typo in TestApp (app_test.go)
  In dashboard/app/app_test.go, the TestApp function contains a ReportingUpdate call with a hardcoded string literal "id" instead of a valid reporting ID:

   1 // dashboard/app/app_test.go:908
   2 c.globalClient.ReportingUpdate(&dashapi.BugUpdate{
   3     ID:         "id", // Should likely be rep.ID or similar
   4     Status:     dashapi.BugStatusOpen,
   5     ReproLevel: dashapi.ReproLevelC,
   6 })
  This causes the API to return an "internal error (failed to find bug by reporting id 'id')", which triggers a t.Fatalf in the test's error handler.

  2. Race Condition in TestAIJobParallelPoll (ai_test.go)
  TestAIJobParallelPoll failed because multiple concurrent agents were assigned the same job ID. This indicates a race condition in the aidb.StartJob transaction logic
  or an issue with the Spanner emulator's isolation during concurrent UNNEST queries. The logs show two different agents receiving ID:
  "23b70e22-df2f-49e8-a16b-d89c2d3bd57e", causing the assignedJobs counter to exceed the expected value of 1.

  3. Outdated Test Data in AI Tests (ai_test.go)
  The TestAIJobActions and TestAIJob tests in dashboard/app/ai_test.go use an outdated results format:

   1 // dashboard/app/ai_test.go:308
   2 Results: map[string]any{"Patch": "patch"},
  However, the ai.PatchingOutputs struct (defined in pkg/aflow/ai/ai.go) was recently updated and now uses PatchDiff and PatchDescription instead of Patch. Because the
  dashboard unmarshals these results using DisallowUnknownFields, it fails with: failed to unmarshal ai.PatchingOutputs: json: unknown field "Patch".

  4. Namespace Isolation Failure in ReportingPollBugs
  The logs show reporting_poll_bugs failing with the error must not be called within a namespace. This occurs because the globalHandler for this API detected a non-empty
  namespace (ns "test1"). This suggests that either the globalClient in the tests is being incorrectly associated with the test1 namespace or there is a misconfiguration
  in testConfig where the reporting client is erroneously added to a specific namespace.

UPD:

Number 4 is WAI: the messages were from the tests that tested specifically that.
Sent #6995 to address 1 and 3.

[tarasmadan]

@gemini-cli do the root cause analysis of the problem

[github-actions]

🤖 Hi @tarasmadan, I've received your request, and I'm working on it now! You can track my progress in the logs for more details.

[github-actions]

🤖 AI Assistant: Plan of Action

I have analyzed the request and propose the following plan. This plan will not be executed until it is approved by a maintainer.

Resource Estimate:

• Estimated Tool Calls: ~2
• Files to Modify: 1

Root Cause Analysis:

The flaky test TestAIJobParallelPoll is caused by a race condition in dashboard/app/aidb/crud.go within the StartJob and NextStaleJob functions. These functions rely on Google Cloud Spanner's client.ReadWriteTransaction.

Cloud Spanner handles write conflicts by aborting the conflicting transaction and automatically retrying the provided function closure. During concurrent polling (like the 15 concurrent requests in the test):

1. Multiple transactions might query and read the exact same available job, assigning it to the outer-scoped variable job = jobs[0].
2. All transactions try to commit the update. One succeeds.
3. The other transactions abort with a concurrent modification error and the Spanner client automatically retries their closures.
4. On the retry attempt, the query returns 0 results because the job was already taken. The closure returns a nil error because of the logic: if err := spanner.SelectAll(iter, &jobs); err != nil || len(jobs) == 0 { return err } (where err is nil).
5. The transaction completes successfully from the client's perspective.
6. The Bug: Because the outer-scoped variable job is never reset during the retry, it retains the Job object from the aborted attempt. The function then incorrectly returns this already-claimed job instead of nil, causing multiple workers to think they acquired the same job.

Proposed Steps:

• Step 1: Update dashboard/app/aidb/crud.go to explicitly reset job = nil at the very beginning of the client.ReadWriteTransaction closures in both StartJob and NextStaleJob. This ensures the captured state is correctly cleared across automatic retry attempts.

Please review this plan. To approve, comment @gemini-cli /approve on this issue. To make changes, comment changes needed.