Embracing the Bitter Lesson: Scaling Compute and Energy for AI Progress

In February 2025, I had a significant shift in perspective at OpenAI. Reinforcement Learning (RL) was transforming how we approached problems where verification is easier than generation. The Strawberry team was achieving remarkable results by scaling up RL compute, saturating their environments every few weeks. This experience led me to ponder the highest-value RL environment and reaffirmed the importance of Richard Sutton's "Bitter Lesson."

The Bitter Lesson Revisited

Sutton’s “Bitter Lesson” emphasizes that general methods leveraging computation are the most effective in AI research. Moore's Law, or its broader implication of exponentially falling computational costs per unit, underpins this lesson. Despite this knowledge, many researchers still focus on algorithms, architecture, and data as if scaling laws were a recent discovery.

The Compute-First Approach

The fundamental insight is that more compute and energy are the most reliable paths to advancing AI. This doesn’t mean ignoring algorithmic improvements; rather, it suggests prioritizing scalable solutions. Here’s what this means in practice:

• Algorithmic Improvements: While crucial, they often provide diminishing returns compared to raw computational power.
• Data Efficiency: Enhancing data efficiency can reduce the need for massive datasets but is still limited by available compute.
• Scalable Architectures: Designing models that can effectively utilize increasing amounts of compute and data.

Recursive Self-Improvement

The dream at many frontier labs is recursive self-improvement: AI systems capable of coding better versions of themselves, leading to exponential intelligence growth. However, this notion often overlooks the Bitter Lesson. Research is compute-bound, and even with advanced AI, the bottleneck remains physical constraints.

Real-World Science as the Bottleneck

To truly take the Bitter Lesson seriously, we must focus on accelerating the technologies that are currently bottlenecked by real-world science:

• Compute Hardware: Advances in GPU technology (like Nvidia’s H100) and manufacturing processes (ASML’s High-NA EUV).
• Energy Efficiency: Improvements in energy sources and consumption, such as advancements in nuclear fusion.

Practical Steps

Here are some practical steps to align with the Bitter Lesson:

• Invest in Compute Infrastructure: Allocate resources to build and maintain robust compute clusters.
• Collaborate on Hardware Development: Partner with hardware manufacturers to drive innovation.
• Energy Research: Fund and support research into sustainable and efficient energy sources.

Case Study: OpenAI’s Strawberry Team

The Strawberry team at OpenAI exemplifies the power of scaling compute. By continuously increasing the computational resources for RL, they achieved rapid progress in various environments. This approach demonstrates that with sufficient compute, even complex problems can be systematically solved.

Conclusion

Taking the Bitter Lesson seriously means acknowledging that more compute and energy are essential for AI advancement. While algorithmic improvements and data efficiency are important, they should not overshadow the need for scalable solutions. By focusing on real-world science to accelerate compute and energy technologies, we can pave the way for true recursive self-improvement and a future where intelligence overflows.