Anthropic Identifies Industrial-Scale Distillation Attacks by AI Labs

Anthropic, a leading AI research laboratory, has uncovered industrial-scale campaigns by three other AI labs-DeepSeek, Moonshot, and MiniMax-to illicitly extract capabilities from its model, Claude. These campaigns involved over 16 million exchanges through approximately 24,000 fraudulent accounts, violating Anthropic’s terms of service and regional access restrictions.

Why It Matters

The use of distillation-a technique where a less capable model is trained on the outputs of a more powerful one-can be both legitimate and illicit. While frontier AI labs often use distillation to create smaller, cheaper versions of their models for customers, it can also be exploited by competitors to acquire advanced capabilities at a fraction of the development time and cost. This illicit activity poses significant risks to national security and global stability.

Key Risks

Illicitly distilled models lack the necessary safeguards that are built into legitimate AI systems. Anthropic and other U.S. companies implement robust measures to prevent state and non-state actors from using AI for malicious purposes, such as developing bioweapons or conducting cyber attacks. However, models created through illicit distillation are unlikely to retain these protections, leading to the proliferation of dangerous capabilities.

Foreign labs that successfully distill American models can integrate these unprotected capabilities into military, intelligence, and surveillance systems. Authoritarian governments could then deploy frontier AI for offensive cyber operations, disinformation campaigns, and mass surveillance. If distilled models are open-sourced, the risk multiplies as these capabilities spread freely beyond any single government's control.

The Opportunity

Addressing the threat of distillation attacks requires rapid, coordinated action among industry players, policymakers, and the global AI community. Anthropic has consistently supported export controls to maintain America’s lead in AI development. These controls are designed to prevent foreign labs, particularly those under the influence of the Chinese Communist Party, from closing the competitive gap through illicit means.

However, without visibility into these attacks, the apparent rapid advancements by foreign labs may be misinterpreted as evidence that export controls are ineffective. In reality, much of this progress depends on capabilities extracted from American models. By identifying and countering distillation attacks, the industry can better protect intellectual property and maintain a strategic advantage.

Conclusion

The threat of industrial-scale distillation attacks is real and growing. Anthropic’s discovery underscores the need for immediate and coordinated action to safeguard AI technology and prevent its misuse. The global AI community must work together to implement robust defenses and ensure that the benefits of AI are realized while minimizing the risks.