OpenAI Launches o1-preview: AI Models with Enhanced Reasoning Capabilities

---

Introducing OpenAI o1-preview

September 12, 2024

OpenAI has just released the first in a new series of AI models designed to spend more time thinking before they respond. These models, named o1-preview, are specifically trained to reason through complex tasks and solve harder problems than previous iterations in fields like science, coding, and math.

What Changed Technically?

• Enhanced Reasoning: The key technical advancement is the training methodology that encourages these models to think more deeply before responding. This involves refining their thinking process, trying different strategies, and recognizing mistakes.
• Performance Improvements:
  • In physics, chemistry, and biology benchmark tasks, the next model update performs similarly to PhD students.
  • For math problems, particularly in the International Mathematics Olympiad (IMO), GPT-4o solved only 13% of problems correctly, while the reasoning model scored 83%.
  • In coding competitions like Codeforces, the new model reached the 89th percentile.

How It Works

The models are trained to spend more time on problem-solving, much like a human would. Here’s a breakdown of the training process:

• Refinement: Models learn to refine their thinking by iterating through different approaches.
• Strategy Testing: They try various strategies to solve problems, which helps in finding the most effective solution.
• Error Recognition: The ability to recognize and correct mistakes is crucial for improving accuracy.

Safety Enhancements

OpenAI has also introduced a new safety training approach that leverages the models' reasoning capabilities. This ensures they adhere to safety and alignment guidelines more effectively:

• Contextual Reasoning: Models can reason about safety rules in context, making them better at applying these rules.
• Jailbreaking Resistance: In jailbreaking tests (where users try to bypass safety rules), GPT-4o scored 22 out of 100, while the o1-preview model scored 84. This significant improvement highlights the effectiveness of the new training approach.

Rate Limits and Availability

• Rate Limits:
  • As of September 17, 2024, rate limits for o1-preview are set to 50 queries per week.
  • For o1-mini, the limit is 50 queries per day.
• Availability: The model is currently available as a preview in ChatGPT and through the OpenAI API. Regular updates and improvements are expected.

Key Takeaways for Practitioners

• Complex Reasoning Tasks: If your work involves complex reasoning tasks, o1-preview offers significant advancements over previous models.
• Safety and Alignment: The new safety training approach ensures better adherence to guidelines, making it a more reliable choice for sensitive applications.
• Feature Set: While the model is still in early stages and lacks some of ChatGPT's features (like web browsing and file uploads), its reasoning capabilities are a major step forward.

Future Updates

OpenAI plans to release evaluations for the next update, which is currently in development. This will provide more insights into the model’s performance and potential applications.

For more detailed information, you can refer to OpenAI's technical research post and the system card.