I Shrunk Mistral's LARGEST 1T AI to Run Locally... it's actually GREAT! 🇫🇷

Summary

The video tests whether a quantized version of Mistral’s roughly 1-trillion-parameter model can run locally and produce useful results. The host runs it on a 512 GB Mac Studio, with about 52 billion active parameters. The model reportedly requires around 430 GB of memory, leaving limited room for context and other tasks. A simple question about Paris runs at roughly 15 tokens per second.

Setup, licensing, and quantization

The host says local use is permitted under the model’s license, but offering it as a competing hosted service may require a commercial license.

To fit the model on the machine, the host compares standard Q3 quantization with a modified Q3 INF version. In the host’s tests, standard Q3 sometimes produces execution or shader errors when generating complex code, while Q3 INF is more reliable in several demonstrations.

The host cautions that long generations and quantization can compound errors: a small change in token selection can lead to incorrect code or output. Higher “thinking” settings also generate more tokens, increasing the chances of issues.

Generation and coding demonstrations

The host tests several projects:

• Tetris: Works reasonably well.
• 3D volcano and lava-flow simulation: Standard Q3 encounters problems, while Q3 INF produces a working scene with falling buildings, lava, ash, and a day/night transition.
• 3D helicopter: Can take off, land, explode, and be disassembled and reassembled. The host reports no execution errors in the Q3 INF generation.
• Fat-cat game: A cat hunts mice. The local version includes music, sound effects, a hunger meter, and mouse-catching mechanics, though it has control and visual quirks.

The host compares the local cat game with one generated through Mistral’s cloud interface. The cloud version took under ten minutes, compared with about an hour and a half locally in the host’s debugging setup. The local version uses basic HTML, while the cloud output uses React. The cloud game has a more elaborate introduction and movement, but lacks some features of the local version, such as music and a hunger meter. Both versions have bugs, including reversed movement controls. The host also objects to the cloud generation using the channel’s username in the project name despite privacy settings.

Other creative tests include:

• 3D human-face generator: Initially fails at runtime, but the host fixes it by correcting a small code issue. The interface allows adjustments such as lighting, skin tone, hair density, hairstyle, and rotation, though some features—such as eye rendering—do not work as intended.
• Music-generation app: Produces music the host likes, allows tempo changes, and successfully exports a WAV file.
• Cat-themed marketing website: Includes interactive elements such as a 3D cat, an adoption button, and animated page sections. The host sees it as a strong use case for quickly creating product-launch pages.
• Model infographic: Looks attractive, but invents specifications and misstates licensing and memory requirements. The example shows that visual quality does not guarantee factual accuracy.

Reasoning and programming tests

The model answers some straightforward questions correctly, but results vary with the prompt and thinking level. In a car-wash riddle, it initially recommends walking; with more context and a higher thinking setting, it notices that the car wash is in Greece while the user is in Germany. It also answers a surgeon-and-father riddle correctly.

The lyrics tests are mixed. The model initially refuses to provide copyrighted lyrics, but when prompted to continue with a translation, it invents lyrics instead of translating the song. It also fails to count a repeated word in a song excerpt, responding poetically instead. The host finds the model creative, but not reliably accurate.

For a Windows Store/WinRT programming question, the model identifies a product-type issue and gives the expected answer. The host says it answers correctly with relatively few tokens and contrasts this with another model that needs a higher thinking setting for the same question.

Overall assessment

The host is impressed by the quality of the local demonstrations, particularly the website, music app, and cat game. The host says Q3 INF performs much more reliably than standard Q3 in the examples shown. Local execution is far slower than cloud generation, but can still produce capable and sometimes comparable results.

The tests also reveal practical limitations: code-generation failures, inconsistent reasoning, hallucinated facts, and bugs in generated apps. The video’s benchmark discussion is brief, and the host expresses uncertainty about how one of the reported intelligence scores was calculated.

Main speaker or source

• The video host from the xCreate channel provides the demonstrations, local-versus-cloud comparison, and evaluations. No other speaker is clearly identified.