Moonshot's Kimi K3.1 Pro Tops the Benchmarks, But the Gains Are Workload-Specific

Moonshot dropped its biggest model yet on Friday, and the benchmark numbers are the kind that make you do a double take. Kimi K3.1 Pro is a 3-trillion-parameter post-trained version of Kimi K3, the open-source model Moonshot released back in July. According to both Moonshot and third-party testing firm Artificial Analysis, it's good enough to take the top open-weights spot in the world once the weights actually drop, which Moonshot says will happen November 4.

Right now it's API-only. But the scores already put it ahead of Xiaomi's MiMo-V2.6-Pro, DeepSeek V4.1-Flash-1017, Meta's Muse Wonder, and even OpenAI's GPT-6 Astra on Artificial Analysis's Intelligence Index. That's a notable list of names to leapfrog in one release.

Here's the part that matters for anyone building agents: the biggest gains aren't in raw knowledge tests. They're in tasks that require doing a sequence of things correctly, not just answering a hard question once.

The Agentic Jump

Look at Terminal-Bench 4.0, a benchmark for agents that have to inspect files, edit code, run tests, and recover from failures in a terminal environment. Kimi K3 scored 13%. K3.1 Pro scored 67%, ahead of Claude Sonnet 5.5's 64%. That's a 54 percentage point jump on a benchmark specifically designed to punish models that can't handle iterative, tool-using workflows.

The pattern holds across other agentic evals:

• CyberGym-E2E-AA: 58% to 95%, edging out Daybreak Blue's 93%
• Cyber Index: 41% to 64%, ahead of DeepSeek's 57%
• AutomationBench-AA: 58% to 73%, a 15-point gain on business-software workflows
• GDPval-AA: 1,534 to 1,846 Elo, third place behind Opus 5.5 (1,866) and Wonder Full (1,857)
• Intelligence Index overall: 44 to 55, good for fourth place

Compare that to the more static reasoning benchmarks, where gains are real but modest: Humanity's Last Exam goes from 47% to 52%, CritPt physics from 23% to 25%, SciCode from 59% to 61%. The story here isn't "Kimi got smarter across the board." It's "Kimi got dramatically better at executing multi-step tasks," which is a narrower but arguably more commercially useful claim.

Artificial Analysis's head-to-head comparison against Opus 5.5 backs this up: Kimi wins on AutomationBench-AA (73% vs 70%) and document reasoning (31% vs 26%), but Opus still leads on Humanity's Last Exam (61% vs 52%) and CritPt (32% vs 25%). Pick your model by workload, not by the aggregate leaderboard position.

Moonshot's own benchmark table tells a similar but not identical story. It has Kimi beating both Opus and Astra on SWE-Marathon (51.7 vs 46.3 and 42.5), Agents' Last Exam, and financial-document analysis via OfficeQA Pro v2. But on Terminal-Bench Science 0.1, a scientific-computing variant, Kimi trails badly: 38.1 versus Opus's 63.3 and Astra's 68.1. And on BenchCAD 3D reconstruction, it's behind both. Interestingly, Moonshot's own Terminal-Bench 4.0 number (65.7, narrowly behind Opus's 66.4) doesn't match Artificial Analysis's independently measured 67% lead. These aren't contradictory so much as a reminder that different test harnesses and agent scaffolding produce different numbers, even for the same nominal benchmark.

On pricing, Kimi K3.1 Pro lands at $2 per million input tokens and $10 per million output tokens, roughly a third cheaper than K3. That undercuts Opus 5.5 and Muse Wonder by half and beats Astra's base rate by 80%. It's tied with Sonnet 5.5 on input/output pricing, though Sonnet scores a point higher on the Intelligence Index and charges less for cached input. Against DeepSeek at peak rates, Kimi still costs about 6.7x more per input token and 8.3x more per output token, so the cheap-open-model narrative only holds relative to the other frontier-tier players, not against DeepSeek's budget tier.

There's also a Highspeed variant at $4/$20 that Moonshot claims runs two to three times faster at equivalent intelligence, useful if your bottleneck is latency rather than per-token cost.

On the security side, Moonshot says it deliberately filtered offensive-capability training data and RL tasks. The unfiltered internal version solved 90.5% of exploit-development problems versus 81% for the shipped model, a real safety tradeoff Moonshot is transparent about. Component scores on the Cyber Index also vary by task: Kimi leads CyberGym-E2E-AA at 95% but trails DeepSeek on CWE-Bench-AA (61% vs 69%) and Daybreak Blue on DeepsecBench-AA (37% vs 48%). Worth noting too that Artificial Analysis reports zero safety refusals from Kimi on these cyber tasks, which inflates its score somewhat since some competing services lose points for declining requests outright.

One demo stands out as a genuine engineering story. Moonshot built a walking-simulator pipeline around human-atlas, an anatomical model with 296 skeletal elements and 402 muscle components. Across 167 missions and 420 reviews, the simulated body went from falling within seconds to alternating steps for 11.2 seconds. When reward tuning alone couldn't fix arm movement, the model dug into the simulator itself and found 36 shoulder-muscle moment-arm calculation errors, then fixed them and retrained. That's a debugging instinct worth looking for in engineering agents: investigate the environment before endlessly tweaking the policy. It's one demo, not proof of generalization, but it's the kind of anecdote that tells you something about training philosophy.

All of this launches alongside an allegation, circulating on X, that Moonshot paid influencers to hype the release. Nothing's been independently verified there, but it's the kind of thing that tends to follow splashy benchmark wins in this industry, and it's worth keeping in the back of your mind when reading the glowing early takes.

Key Takeaways

Kimi K3.1 Pro's headline numbers are real but narrow: massive gains on agentic, tool-using, iterative-coding benchmarks, more modest gains on static reasoning tests, and outright weaknesses in scientific computing and some cyber subcategories. Pricing is competitive against Opus and Muse Wonder but not against DeepSeek. Teams evaluating it should pilot on the specific workload (terminal-based coding, business-software automation, financial document analysis) rather than treating the aggregate Intelligence Index ranking as a universal verdict. Open weights land November 4, which is when the real independent scrutiny starts.