Sky-T1-32B-Preview: Affordable and Open-Source Reasoning Model Trained for Under $450

Sky-T1-32B-Preview: Affordable and Open-Source Reasoning Model Trained for Under $450

The NovaSky team at UC Berkeley has introduced Sky-T1-32B-Preview, a reasoning model that matches the performance of O1-preview on popular benchmarks. What sets Sky-T1 apart is its affordability; it was trained for less than $450, making high-level reasoning capabilities accessible to a broader audience.

Technical Overview

Key Achievements

• Performance Parity: Sky-T1-32B-Preview performs on par with O1-preview in both reasoning and coding tasks.
• Cost Efficiency: Trained for under $450, demonstrating that advanced models can be developed affordably.
• Open-Source: All code, data, and model weights are publicly available.

Why It Matters

Models like O1 and Gemini 2.0 have shown remarkable capabilities in reasoning by generating long internal chains of thought. However, these models often come with proprietary constraints, limiting access to the academic and open-source communities. Sky-T1-32B-Preview addresses this gap by providing a fully transparent and accessible alternative.

Open-Source Collaboration

To drive progress together, NovaSky has made all resources available:

• Infrastructure: A single repository for data preparation, training, and evaluation.
• Data: 17K curated data points used to train Sky-T1-32B-Preview.
• Technical Details: A comprehensive technical report with detailed documentation and a Weights & Biases (W&B) log.
• Model Weights: The 32B model weights are available on Hugging Face.

Comparison with Other Models

| Model | Sky-T1-32B-Preview | STILL-2 | Journey | QwQ | O1 | | --- | --- | --- | --- | --- | --- | | Data | ✅ | ✅ | ❌ | ❌ | ❌ | | Code | ✅ | ❌ | ❌ | ❌ | ❌ | | Report | ✅ | ✅ | ✅ | ❌ | ❌ | | Math Domain | ✅ | ✅ | ✅ | ✅ | ✅ | | Coding Domain | ✅ | ❌ | ❌ | ✅ | ✅ | | Model Weights | ✅ | ✅ | ❌ | ✅ | ❌ |

Data Curation Process

To generate the training data for Sky-T1-32B-Preview, we used QwQ-32B-Preview, an open-source model with reasoning capabilities similar to O1-preview. The data curation process involved several key steps:

• Data Mixture: We curated a diverse set of domains that require reasoning.
• Reject Sampling: This procedure was employed to improve the quality of the training data by filtering out low-quality samples.
• Rewriting: QwQ traces were reformatted using GPT-4o-mini to ensure they were well-structured and consistent.

Technical Details

Training Infrastructure

• Repository: All necessary scripts and configurations are available in a single repository, making it easy for others to replicate the training process.
• Data Sources: The 17K data points cover a wide range of reasoning tasks, ensuring the model is well-rounded.
• Training Costs: The total cost for training Sky-T1-32B-Preview was under $450, thanks to efficient use of resources and optimized training techniques.

Evaluation

• Benchmarks: Sky-T1-32B-Preview was evaluated on popular reasoning and coding benchmarks, demonstrating performance parity with O1-preview.
• W&B Log: Detailed logs are available for transparency and reproducibility.

Conclusion

Sky-T1-32B-Preview is a significant step forward in making advanced reasoning models accessible and affordable. By open-sourcing all components, NovaSky aims to empower the community to build on this work and explore new frontiers in AI research.