GPD-1: A Unified Transformer Model for Autonomous Driving Tasks

The field of autonomous driving has seen a surge in interest, particularly around the challenges of scene evolution and decision-making. Most existing methods tackle one aspect of this problem, such as map generation, motion prediction, or trajectory planning. However, a new paper introduces GPD-1 (Generative Pre-training for Driving), a unified model that can handle all these tasks without additional fine-tuning. This is significant because it simplifies the development and deployment of autonomous driving systems by reducing the need for specialized models.

Key Technical Changes

• Unified Token Generation Problem: GPD-1 represents each scene using ego (vehicle), agent (other vehicles, pedestrians), and map tokens. This approach allows the model to treat autonomous driving as a unified token generation problem.
• Autoregressive Transformer Architecture: The model uses an autoregressive transformer with a scene-level attention mask. This enables bi-directional interactions within scenes, which is crucial for capturing complex dependencies between different elements in the driving environment.
• Hierarchical Positional Tokenizer: For ego and agent tokens, GPD-1 introduces a hierarchical positional tokenizer that effectively encodes 2D positions and headings.
• Map Vector-Quantized Autoencoder (VQ-VAE): The map tokens are generated using a VQ-VAE to compress semantic maps into discrete tokens. This compression is essential for efficient processing and generalization.

Model Architecture

Scene Representation

• Ego Tokens: Represent the vehicle's state, including position and heading.
• Agent Tokens: Capture the states of other vehicles and pedestrians in the scene.
• Map Tokens: Encode the semantic map information, such as road layouts and obstacles.

Token Generation

• The autoregressive transformer generates tokens one by one. During training, it conditions on ground-truth tokens, while during inference, it uses previously predicted tokens.
• The scene-level attention mask ensures that the model can consider all relevant interactions within a scene, making it more robust to dynamic environments.

Key Innovations

1. 2D Map Scene Tokenizer:
   • VQ-VAE: Trains a vector-quantized autoencoder to compress ego-centric semantic maps into discrete tokens.
   • This allows the model to efficiently handle large-scale map data and maintain spatial coherence.

2. Hierarchical Quantization Agent Tokenizer:
   • Thresholds for Categorization: Uses predefined thresholds to categorize agent states (e.g., speed, direction) into distinct ranges.
   • Converts continuous information into discrete tokens, which simplifies the model's input and improves computational efficiency.

Pre-training and Evaluation

• Dataset: GPD-1 is pre-trained on the large-scale nuPlan dataset, which contains diverse driving scenarios.
• Experiments: Extensive experiments demonstrate the model's effectiveness across various tasks:
  • Scene Generation: Generates complete driving scenes from partial inputs.
  • Traffic Simulation: Simulates traffic flow and interactions between agents.
  • Closed-loop Simulation: Evaluates the model's performance in a closed-loop environment, where predictions are fed back into the system.
  • Map Prediction: Predicts future map configurations based on current observations.
  • Motion Planning: Plans trajectories for ego vehicles and other agents.

Results

• Generalization: GPD-1 successfully generalizes to various tasks without fine-tuning, showcasing its versatility and robustness.
• Performance: The model outperforms specialized models in several benchmarks, indicating its potential for real-world applications.

Given past 2D bird's-eye-view (BEV) observations, the pre-trained GPD-1 model can predict future scene evolution and agent movements. This capability is crucial for autonomous driving systems that need to make informed decisions based on dynamic and complex environments.

Conclusion

GPD-1 represents a significant step forward in the development of unified models for autonomous driving. By treating driving tasks as a token generation problem, it simplifies the model architecture while maintaining high performance across multiple tasks. The use of advanced tokenizers and an autoregressive transformer with scene-level attention makes GPD-1 a promising tool for both research and practical applications.