dashboard: restart abandoned AI jobs#6889
Merged
a-nogikh merged 4 commits intogoogle:masterfrom Mar 10, 2026
Merged
Conversation
Collaborator
|
Please show how these new tables/columand will be used in the end. W/o that it's hard to understand if it's the right approach or not. |
a-nogikh
force-pushed
the
features/ai-job-restart-prepare
branch
from
8ef6499 to
da271e5
Compare
a-nogikh
changed the title
Collaborator
Author
|
Added the actual logic. We indeed also recreated automatic jobs in |
dvyukov
reviewed
Mar 6, 2026
a-nogikh
force-pushed
the
features/ai-job-restart-prepare
branch
from
da271e5 to
08987dc
Compare
dvyukov
reviewed
Mar 9, 2026
For now, use a dashboard client name. In a prod deployment, that would be a persistent name of the machine where the syz-agent is hosted. This information be used later for restarting the aborted jobs.
a-nogikh
force-pushed
the
features/ai-job-restart-prepare
branch
from
08987dc to
b6e1288
Compare
Update the timestamp on each job poll or each trajectory update.
If an agent has taken a job and then, without finishing it, comes for the next one, recreate that job. If an agent has started working on a job, not finished it and completely disappeared for more than 8 hours, give the job to another agent. At the same time, don't look at the stale jobs in the automatic job creation logic - let it focus on recreating explicitly failed jobs.
Make Workflows be interleaved into Agents. This enables a more complete data representation and enables a more precise tracking of the active workflow types.
a-nogikh
force-pushed
the
features/ai-job-restart-prepare
branch
from
b6e1288 to
23df962
Compare
dvyukov
approved these changes
Mar 10, 2026
Collaborator
Author
|
Deployed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cc #6781.
The changeset prepares the AI dashboard for the automatic restart of the aborted jobs. When suggesting the restarted job, we need to give preference to the agent which was already executing it (to reuse its cache), so we need to remember more information in the database.
The agent name is currently equal to dashboard client, but later we'd take this name elsewhere.