Video summary

They actually did it... (Kimi K3.1)

Main summary

Key takeaways

Technology

Summary

Matthew Berman reviews Kimi K3.1 Pro, describing it as a highly capable open-weights model that approaches leading proprietary models on some benchmarks, though his hands-on tests produce mixed results. The video combines product demonstrations, model comparisons, and a review of Kimi’s published specifications and benchmark claims.

Hands-on demonstrations

  • 3D Rubik’s Cube: A strong result. The browser-based simulation featured realistic lighting and reflections, keyboard and button controls, a timer, undo and reset options, adjustable turn speed, and a scramble function. It solved the cube using a precomputed solution rather than simply reversing the scramble. Berman calls this a clear pass and highlights 3D rendering as a strength.
  • iPhone Duo rendering: Kimi produced a WebGL model with convincing materials and reflections, but it did not accurately represent the device. A second attempt involved web research and took 28 minutes and 45 seconds. Berman planned to check whether a more specific prompt improved accuracy.
  • Belaying-knot tasks: A Blender task combining web search, computer control, and spatial reasoning failed to produce a properly tied knot or attach it correctly to the harness. A motion graphic intended to teach the knot was also not clear enough to follow. A GPT comparison made the rope’s over-under path easier to understand. The video notes that the graphics were presented as visual guides, not substitutes for qualified instruction.
  • Fall Guys-style browser game: “Bean Blitz” included 12 animated competitors, CPU rivals, obstacles, checkpoints, qualification, and adjustable settings. Berman considered it a decent result for an open-weights model, but noted pastel colors, flickering, and unreliable collisions.
  • Website redesign: Kimi retained the site’s stated brand colors and fonts, but the result had spacing issues and what Berman described as a generic AI-designed feel.
  • Video generation and editing: Kimi created a five-minute parody video. It reportedly used Kimi’s video-generation API, Python and NumPy for orchestration and procedural audio, and FFmpeg for editing and encoding. Berman was impressed by the technical execution, while acknowledging that the video itself was deliberately chaotic.

Berman also tested Kimi’s benchmark-chart generation. Its first chart omitted benchmark names, and a later version contained invented placeholder scores. After he supplied published figures and prompted it again, Kimi produced a chart based on its materials. Berman presents this as an example of the need to verify outputs and sometimes provide additional direction.

Specifications and performance claims

The video cites Kimi’s published specifications for K3.1 Pro:

  • 2.8 trillion total parameters—described in the introduction as a three-trillion-parameter-class model—with 104 billion active parameters per prompt, indicating a mixture-of-experts design.
  • A one-million-token context window.
  • Multimodal input, including images. Berman had not tested video and said audio support was likely.
  • Claimed improvements in coding, 3D modeling, CAD, and finance, including reduced hallucinations in financial modeling.

Berman reports strong results on several published evaluations, while noting weaker performance on the science benchmark shown:

  • Terminal Bench: Kimi scored 65.7, close to Opus 5.5 at 66.4.
  • GDPval: Kimi scored 1846 Elo, behind Opus 5.5 at 1866.
  • OSWorld: Kimi scored 79.6, which Berman describes as competitive with leading models in computer-use tasks.
  • Terminal Bench Science: The score shown was substantially below the leading result.
  • Other comparisons included Kimi-created evaluations and third-party tests. Berman cautions that strong benchmark performance does not necessarily translate to equal performance in practical use.

On the Artificial Analysis Intelligence Index, the video places Kimi at 55, behind a small group of leading models but ahead of several others shown. It also highlights a cost-per-intelligence-task estimate of $2.11, compared with $7.63 for the highest-cost model in the chart.

Pricing and overall assessment

The listed API pricing is $2 per million input tokens and $10 per million output tokens at regular speed. The high-speed option is listed at $4 and $20, respectively, and is described as two to three times faster. Cached input is listed at $0.20 per million tokens.

Berman’s overall view is that Kimi K3.1 Pro is a notable open-weights model with especially promising 3D and coding capabilities, as well as competitive benchmark performance for its cost. However, he finds that it can require more prompting than some alternatives, and several demos reveal weaknesses in accuracy, visual clarity, and polish.

He also speculates that Kimi’s resemblance to other leading models in some areas could reflect distillation, but presents this as speculation rather than established fact.

Guides and comparisons featured

  • A belaying-knot visual-instruction test, comparing Kimi’s motion graphic with a GPT-generated version.
  • A Fall Guys-style game-building test.
  • Comparisons against models including GPT 6.1 Soul and Opus 5.5, as named in the subtitles.
  • A review of Kimi’s published benchmark charts, model specifications, and pricing.

Main speakers and sources

  • Matthew Berman — narrator, evaluator, and presenter of the demonstrations.
  • Kimi’s blog post and published benchmark materials — sources for the model specifications, technical claims, and reported benchmark results.
  • CodeRabbit — sponsor featured in a segment about security scanning and code-review workflows.

Rate this summary

Your feedback will help improve summaries.

Improve this summary

Reprocess with a stronger model when the summary feels incomplete or inaccurate.

Pro

Translate summary in another language

Pro

Ask questions to this video

Chat for follow-up questions, clarifications, and source-backed answers.

Coming soon

Share this summary

Original video