DiffusionDrive: A Truncated Diffusion Model for Real-Time End-to-End Autonomous Driving

In a significant leap forward for autonomous driving, researchers from HUST-VL have introduced DiffusionDrive, a novel truncated diffusion model designed to generate diverse and high-quality driving actions in real-time. The paper, titled "DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving," has been accepted as a highlight at CVPR 2025. This work addresses the challenges of applying diffusion models-typically used for generative tasks like image synthesis-to the dynamic and complex environment of autonomous driving.

Why It Matters

Diffusion models have gained traction in robotics due to their ability to model multi-modal action distributions, which is crucial for handling the varied and unpredictable nature of traffic. However, traditional diffusion policies require numerous denoising steps, making them computationally expensive and unsuitable for real-time applications. DiffusionDrive tackles this by truncating the diffusion schedule and incorporating prior multi-mode anchors, reducing the number of denoising steps while maintaining or even enhancing action diversity.

Key Innovations

• Truncated Diffusion Schedule: By truncating the diffusion process, the model significantly reduces the number of denoising steps required. This is achieved by learning to denoise from an anchored Gaussian distribution directly to the multi-mode driving action distribution.
• Multi-Mode Anchors: These anchors serve as a prior for the diffusion process, guiding the generation of diverse and plausible actions. The use of anchors helps in stabilizing the training process and improving the quality of generated actions.
• Cascade Diffusion Decoder: An efficient cascade architecture that enhances interaction with conditional scene context, ensuring that the generated actions are contextually relevant.

Technical Details

• Model Architecture:
  • Encoder: A ResNet-34 backbone is used to extract features from the input images, providing a robust representation of the driving environment.
  • Decoder: The cascade diffusion decoder consists of multiple layers, each refining the generated actions based on the scene context. This architecture ensures that the model can generate diverse actions while maintaining real-time performance.

• Training and Inference:
  • Training: The model is trained using a dataset of driving scenarios, where the multi-mode anchors are derived from expert trajectories. The training process involves learning to denoise from these anchors to the final action distribution.
  • Inference: During inference, the model can generate high-quality actions in just 2 steps, compared to the numerous steps required by vanilla diffusion policies. This reduction in steps is crucial for achieving real-time performance.

Performance Benchmarks

• PDMS (Path Deviation Mean Squared) Score: On the planning-oriented NAVSIM dataset, DiffusionDrive achieves a PDMS score of 88.1 without any additional optimizations or tricks. This sets a new record and demonstrates the model's superior performance in generating diverse and high-quality driving actions.
• Real-Time Performance: The model runs at a real-time speed of 45 FPS on an NVIDIA 4090, making it suitable for deployment in autonomous vehicles.

Qualitative Results

Qualitative results on challenging scenarios further validate the effectiveness of DiffusionDrive. The model consistently generates diverse and plausible driving actions, even in complex and dynamic environments. This robustness is crucial for ensuring safe and reliable autonomous driving.

Future Work

The researchers plan to release the code and pre-trained models, making it easier for other developers and researchers to build upon their work. They also intend to explore further optimizations and applications of diffusion models in robotics and autonomous systems.