Anthropic Unveils Enhanced Claude 3.5 Sonnet and New Haiku Model with Experimental Computer Use

Today, Anthropic has announced significant updates to its AI models, including the upgraded Claude 3.5 Sonnet and a new model, Claude 3.5 Haiku. These enhancements come alongside an exciting experimental feature: computer use, which allows Claude to interact with computers as humans do.

Upgraded Claude 3.5 Sonnet: Industry-Leading Software Engineering Skills

The upgraded Claude 3.5 Sonnet delivers substantial improvements across various benchmarks, particularly in coding and tool use tasks. Here are the key enhancements:

• Coding Performance: The new model improves its score on the SWE-bench Verified benchmark from 33.4% to 49.0%. This places it ahead of all publicly available models, including specialized systems like OpenAI's o1-preview.
• Tool Use: On the TAU-bench for agentic tool use tasks, the model shows a significant improvement from 62.6% to 75.0%.

These enhancements make Claude 3.5 Sonnet a powerful tool for developers and researchers, offering state-of-the-art capabilities in software engineering.

New Claude 3.5 Haiku: Matching Performance at Similar Speed

Claude 3.5 Haiku is the latest addition to Anthropic's lineup. It matches the performance of Claude 3 Opus, their prior largest model, on many evaluations while maintaining a similar speed to the previous generation of Haiku models. Key features include:

• Performance: On par with Claude 3 Opus in various benchmarks.
• Speed: Comparable to the previous generation of Haiku models.

The new model is set to be released later this month, providing users with an additional option for their AI needs.

Experimental Computer Use: A Groundbreaking Capability

Perhaps the most exciting announcement is the introduction of computer use in public beta. This feature allows Claude to interact with computers as humans do-by looking at screens, moving cursors, clicking buttons, and typing text. Key points:

• First-of-Its-Kind: Claude 3.5 Sonnet is the first frontier AI model to offer this capability in public beta.
• Experimental Stage: While it is currently experimental and may be cumbersome or error-prone at times, Anthropic is releasing it early for developer feedback.
• Rapid Improvement Expected: The company anticipates rapid improvements over time based on user input.

Early Adopters and Use Cases

Several companies have already begun exploring the potential of these new capabilities:

• Replit: Using Claude 3.5 Sonnet's computer use and UI navigation to develop a key feature for their Replit Agent product, which evaluates apps as they are being built.
• Asana, Canva, Cognition, DoorDash, The Browser Company: These companies are also leveraging the new features to carry out complex tasks that require multiple steps.

Availability

The upgraded Claude 3.5 Sonnet is now available for all users. Developers can start building with the computer use beta via the Anthropic API, Amazon Bedrock, and Google Cloud’s Vertex AI. The new Claude 3.5 Haiku will be released later this month.

Pricing Update (12/03/2024)

Claude 3.5 Haiku has been revised in pricing to $0.80 per million tokens for input and $4 per million tokens for output, making it more accessible for a broader range of users.

For more information on how to integrate these new features into your projects, check out the Anthropic API documentation.