AutoE2E - End-to-End AI for Self Driving

⭐ Star us on GitHub — your support motivates us a lot!

Free and fully open-source End-to-End AI model

AutoE2E is an open-source End-to-End AI model which enables autonomous driving across highways, arterial roads and city streets using cameras-only, and without reliance on HD-maps.

AutoE2E outputs can be fused with Physics-based sensors such as LIDAR/RADAR to power fully driverless Robotaxi applications, and the basline camera-only model can be used to enable L2++ automotive ADAS applications for point-to-point hands-free navigation.

To learn more about how to participate in this project, please read the onboarding guide

Getting started

Requires Python 3.12 (the pinned PyTorch build has no wheels for 3.13+).

Using make tool

Toggle view

1. Clone and install dependencies

   git clone https://github.com/autowarefoundation/auto_e2e.git
   cd auto_e2e
   make setup                      # CPU torch wheels
   make setup TORCH_CHANNEL=cu118  # or a CUDA build (cu121, ... work too)

2. Verify the install (optional)

   make test

Using plain pip

Toggle view

Clone and install dependencies

git clone https://github.com/autowarefoundation/auto_e2e.git
cd auto_e2e
pip install -r requirements.txt                      # CPU torch wheels
pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cu118  # or a CUDA build (cu121, ... work too)

Without a make tool, you unfortunately cannot verify the install using a test from the Makefile. It is highly recommended to install the tool through a package manager.

Next steps

• Explore the Model folder for the model components, training and inference.
• Follow the Trial Guide to run the inference test on AWS EC2.

Architecture at a glance

AutoE2E takes 7 surround and telephoto cameras plus a rendered map tile, along with egomotion and visual history, and predicts a 6.4s future driving trajectory (acceleration and curvature at 10Hz). See the Model architecture guide for the full inputs, outputs and forward signature.

Performance

Up to ~76 FPS (SwinV2-Tiny, feature-concat fusion, RTX 5080, batch 1). Full per-GPU inference benchmarks covering latency, jitter and VRAM across backbones, fusion modes and batch sizes live in BENCHMARKS.md. Run the benchmarking script to add results for your own GPU.