Thinking Machines Launches Inkling-Small: A Compact, Efficient AI Model with Competitive Performance

Today, Thinking Machines is proud to announce the release of Inkling-Small, an efficient open-weights model that delivers performance on par with its larger counterpart, Inkling, but at just a quarter of the size. This new model is designed to make advanced AI capabilities more accessible and cost-effective for a broader range of applications.

Inkling-Small is a Mixture-of-Experts (MoE) transformer with 276 billion total parameters and 12 billion active parameters. It was trained on NVIDIA GB300 NVL72 systems, ensuring it can handle complex tasks while maintaining efficiency. Like its predecessor, Inkling-Small supports native reasoning over audio and images, variable thinking effort, and a context window of up to 1 million tokens.

Key Performance Highlights

• Parameter Comparison:

  • Total Parameters: 276 billion (Inkling-Small) vs. 975 billion (Inkling)
  • Active Parameters: 12 billion (Inkling-Small) vs. 41 billion (Inkling)

• Benchmarks:

  • Terminal-Bench 2.1: Inkling-Small achieves competitive performance with significantly lower compute requirements.
  • Humanity's Last Exam (Text-Only): Demonstrates strong reasoning capabilities, comparable to larger models.
  • IFBench: Excellent instruction-following performance, maintaining high accuracy while reducing costs.

Performance-Compute Trade-offs

Inkling-Small excels in balancing performance and compute efficiency. Here’s a breakdown of its performance across key benchmarks:

• Terminal-Bench 2.1:

  • Inkling-Small achieves up to 80% of the performance of larger models like Nemotron 3 Ultra and DeepSeek V4 Flash, but with much lower computational costs.
  • For example, it can complete tasks at a cost of around $0.5c per sample, compared to $2c for Inkling.

• Humanity's Last Exam (Text-Only):

  • It performs within the top tier of models in its class, achieving up to 40% accuracy on text-only reasoning tasks.
  • The variable thinking effort feature allows users to fine-tune performance based on their specific needs, balancing cost and output quality.

• IFBench:

  • Inkling-Small maintains high instruction-following accuracy, with a pass rate of over 80%, comparable to models like Qwen3.5-397B-A17B and MiMo V2.5.
  • The estimated cost per sample is around $0.2c, making it highly cost-effective for enterprise use.

Under the Hood

Inkling-Small’s architecture leverages several key innovations to achieve its efficiency:

• Mixture-of-Experts (MoE): This technique allows the model to dynamically activate only a subset of parameters for each task, significantly reducing computational overhead.
  • Each expert is trained to handle specific types of tasks, ensuring that the model can adapt to diverse inputs efficiently.

• Variable Thinking Effort: Users can control the level of effort the model expends on a given task, allowing them to optimize for either speed or accuracy as needed.

  • This feature is particularly useful in scenarios where real-time performance is critical, such as interactive applications and chatbots.

• Native Multimodal Support: The model can process audio and images alongside text, making it versatile for a wide range of applications.

  • This capability is powered by advanced multimodal fusion techniques that integrate data from different sources seamlessly.

In Practice

The launch of Inkling-Small has significant implications for enterprises looking to adopt AI without the high computational costs associated with larger models:

• Cost Efficiency: By reducing the number of active parameters, Inkling-Small can run on less powerful hardware, making it more accessible to organizations with limited resources.

  • For example, small and medium-sized businesses can now leverage advanced AI capabilities without investing in expensive infrastructure.

• Flexibility: The variable thinking effort feature allows for fine-grained control over performance, enabling users to tailor the model to their specific use cases.

  • This is particularly valuable in dynamic environments where the balance between cost and performance needs to be constantly adjusted.

• Security and Compliance: As enterprises increasingly prioritize data security and compliance, models like Inkling-Small can be deployed with additional safeguards.

  • For instance, AI Guardrails from Alibaba Cloud provides real-time protection for LLMs and AI agents, ensuring that the model operates within defined parameters.

What to Watch

As Thinking Machines continues to refine and expand its suite of AI models, several trends are worth keeping an eye on:

• Model Optimization: Expect further improvements in efficiency and performance as research advances.

  • Techniques like sparse activation and quantization could push the boundaries of what’s possible with smaller models.

• Enterprise Adoption: The success of Inkling-Small will likely spur more enterprises to adopt AI, driving innovation and competition in the market.

  • Look for new applications and use cases that leverage the unique capabilities of compact, efficient models.

• Security Enhancements: With increasing concerns about data privacy and security, expect more robust solutions like AI Guardrails to become standard.

  • These tools will play a crucial role in ensuring that AI deployments are secure and compliant with regulatory requirements.

The launch of Inkling-Small marks a significant step forward in making advanced AI accessible and cost-effective. As the landscape continues to evolve, Thinking Machines is well-positioned to lead the way in delivering innovative solutions for enterprises of all sizes.