-
Notifications
You must be signed in to change notification settings - Fork 559
Open
Labels
CICI related changeCI related changeenhancementNew feature or requestNew feature or requesttestingTesting and coverage related issues.Testing and coverage related issues.
Description
🚀 Parallelize TPU CI tests
Motivation
Our TPU CI takes over an hour. Every PR must pass TPU CI for stability and correctness. That makes PR submission slow.
Pitch
- Sample the running time of tests based on recent logs
- Divide the tests into 2-3 groups
- Change the CI logic to run these groups in parallel, similar to how we already do things for CPU and GPU tests.
Additional context
Currently we have 32 TPU CI runners. If the number of PRs grow, we may need to request additional TPU resources.
Metadata
Metadata
Assignees
Labels
CICI related changeCI related changeenhancementNew feature or requestNew feature or requesttestingTesting and coverage related issues.Testing and coverage related issues.