This repository contains information and code for the tool competition on test selection for self-driving cars in simulation.
Self-driving cars (SDCs) are equipped with onboard cameras and various sensors have already demonstrated the possibility of autonomous driving in real environments, leading to great interest in various application domains. However, despite the necessity of systematically testing such complex and automated systems to ensure their safe operation in real-world environments, there has been relatively limited investment in this direction so far.
The SDC Testing Competition is an initiative designed to inspire and encourage the Software Testing Community to direct their attention toward SDCs as a rapidly emerging and crucial domain.
The competition's current focus is on regression testing for test suites executing system tests of SDCs in simulation.
Regression testing involves test selection, prioritization, and minimization.
Clone the repository and fetch large files from Git LFS.
git clone git@github.com:christianbirchler-org/sdc-testing-competition.git
git lfs fetchRead the competition instruction:
cat competition/<YEAR>.md # e.g., cat competition/2026.mdFirst, create a new subdirectory in the associated competition tools directory. E.g., for the competition on test prioritization:
mkdir tools/prioritizers/<myTool>Generate stubs for the interfaces, that need an implementation for the competition.
On the gRPC website is a list of supported languages and instructions to generate stubs for the interfaces defined in the .proto files.
Example with Python:
python -m pip install grpcio grpcio-tools
python -m grpc_tools.protoc -I./tools/prioritizers/<yourTool> --python_out=./tools/prioritizers/<yourTool> --pyi_out=./tools/prioritizers/<yourTool> --grpc_python_out=./tools/prioritizers/<yourTool> path/to/protofile.protoYou can also checkout the sample tools and simply reuse the stubs there without the need to transpile the interface by yourself.
We provide for each competition an evaluation tool that acts as a gRPC client. First, start your tool, which is a gRPC service. Secondly, run the evaluation tool. The output of the evaluation tool should provide a summary of the evaluation metrics.
Use GitHub Discussions for any kind of questions related to the tool competition.
Do not hesitate to ask questions. If something is unclear then it is likely it is unclear for others as well. We appreciate all kind of feedback to make this competition platform as usable as possible.
The important dates for the competition and the current development of this competition platform are defined as GitHub Milestones.
Any kind of contributions (e.g., feature requests, bug reports, questions, etc.) are welcome. Please refer to GitHub Discussions to let the community know about your contributions.
Many thanks to the following contributors who help to build and maintain the repository:
The order is purely chronological.
SDC Testing Competition Platform
Copyright (C) 2025 Christian Birchler
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
- Responsible for the repository: Christian Birchler