The State of Foundation Models, 2025: Scaling, Economics, and New Paradigms

Generative AI has officially gone mainstream. According to recent data, one in eight workers worldwide now uses AI every month, with a staggering 90% of this growth occurring in just the last six months. This surge has propelled AI-native applications into the billions of annual run rate, marking a significant milestone in the adoption and practical application of AI.

Scaling Continues Across All Dimensions

All technical metrics for foundation models continue to improve at an astounding pace, often more than 10x year-over-year. Key areas of improvement include:

• Cost: The cost of training and running models is decreasing rapidly.
• Intelligence: Models are becoming smarter and more capable.
• Context Windows: The amount of context a model can handle has expanded significantly.
• Task Duration: The average duration of human tasks that a model can reliably perform is doubling every 7 months.

Confusing Economics

The economics of foundation models, however, remain perplexing. While companies like OpenAI and Anthropic are experiencing unprecedented growth, with revenues accelerating to over $1 billion annually, the costs associated with training these models are astronomical. End-to-end training for frontier models can cost nearly $500 million. Moreover, due to intense competition and rapid open-source convergence, models often become obsolete within three weeks of their launch.

Reasoning Models: The New Frontier

One of the most exciting developments is the emergence of reasoning models-AI systems trained to "think before they speak." These models are likely to represent a new scaling law, but training them requires significant advancements in post-training techniques, including:

• Reinforcement Learning: Teaching models to learn from feedback and rewards.
• Reward Models: Developing sophisticated reward functions to guide model behavior.

Post-training may become more important than pre-training as these techniques mature.

AI Infiltrates Specialist Professions

AI copilots and agents are now tackling high-value tasks in virtually all knowledge worker domains, including:

• Engineering
• Accounting
• Design
• Law

These AI tools are not just辅助性的; they are becoming integral to the workflow of professionals across various industries.

Agents: Mainstream but Early

Agents have finally hit the mainstream, with widespread adoption in both consumer and enterprise settings. However, the design patterns and system architectures for building effective AI products are still in their early stages. This presents a significant opportunity for innovation and improvement.

The Future of "AI-Native" Organizations

The rise of generative AI is reshaping organizational structures. Flatter teams of capable generalists will become the norm as AI diminishes the value of specialized skills. Roles such as product management, design, and engineering are likely to blur, leading to more interdisciplinary collaboration.

Setting the Stage: Key Insights

Two key insights have driven this technology wave:

1. Self-Supervised Learning: This approach scales data by training models on vast amounts of unlabeled data. For example:

   • Input: "John went to the mall and bought..."
   • Output: "...a hamburger from Johnny Rockets"

2. Attention Architecture (Transformers): These models are compute-efficient and parallelizable, allowing them to understand full context. For instance:

   • Input: "The dog went to the pound"
   • The model can correctly interpret that "pound" refers to a place for stray dogs, not a unit of weight.

Scaling Models Leads to Emergent Behavior

Google Research in 2022 demonstrated that scaling models leads to emergent behavior-capabilities that were not explicitly programmed but emerge from the complexity of the system. This has driven exponential growth in model size and capabilities.

Fastest Rate of Adoption

The adoption rate of new technology has never been faster. ChatGPT, for example, reached 100 million users in just 60 days, a testament to the rapid acceptance and utility of generative AI.

As we continue to push the boundaries of what foundation models can do, the future looks promising but also complex. The challenge now is to navigate these changes effectively and responsibly, ensuring that the benefits of AI are widely shared.