How This Tiny Model Solved Transcription

Summary

Whistle is a 17 MB speech-to-text model from Cactus Compute, an eight-person startup. It runs locally on a CPU without an internet connection or graphics card, and supports English, German, French, Spanish, Italian, Dutch, and Polish. The video presents it as a compact option for private, low-cost transcription on devices ranging from computers and phones to watches and Raspberry Pi.

How it works and stays small

• Whistle turns audio into a spectrogram, which its encoder processes in chunks. Its decoder generates text piece by piece while checking it against the audio.
• The downloadable model uses two-bit quantization to compress most of its roughly 55 million values. According to the video, the uncompressed version is about 220 MB.
• About a third of the model consists of lookup tables rather than components that perform calculations, helping reduce computation.
• A lighter decoder can use two layers instead of eight. It runs about twice as fast but makes around 60% more errors.
• Before processing, Whistle checks whether the audio is nearly silent and returns nothing if it is. This avoids the kind of hallucinated transcription that the video says can occur with Whisper on silence.

Performance and accuracy

• On an M4 Pro Mac, the video reports that Whistle produces its first token in about 11 ms, versus 73 ms for Whisper base, and decodes about five times faster.
• A browser benchmark cited in the video found Whistle three times faster than Whisper base while requiring a download 13 times smaller. On a phone browser, it reportedly runs at more than twice real time.
• On the LibriSpeech audiobook test, Whistle made about 43 errors per 1,000 words, compared with 49 for Whisper base. The video also says it outperformed Moonshine on that test.
• These results do not hold across all conditions. Whisper base reportedly performs better on meetings and recorded speeches, and has higher average performance across the seven supported languages. Whisper’s largest model is substantially more accurate on the audiobook test, but is about 180 times larger than Whistle.
• The presenter cautions that much of the benchmark data comes from Cactus’s own tests. A cited third-party German comparison favored Whistle over Whisper base.

Features and ways to use it

• The model can return word-level timestamps and confidence values, identify the language, and produce audio embeddings for comparison or search.
• Developers can specify a language, enable real-time streaming, and provide keywords—such as names, product codes, or room names—to improve recognition of domain-specific terms. The video reports that a list of 100 keywords reduced errors on those words by about three-quarters without worsening recognition of other words.
• Streaming mode repeatedly transcribes incoming audio and keeps words that appear consistently across consecutive passes. Longer recordings are split into chunks.
• Whistle is available under the Apache 2.0 license, and Cactus provides an engine for 17 platforms. The video describes a simple Python setup: install the package and run a transcription call. It also mentions microphone-based and side-by-side comparison examples.
• Whistle can be combined with Cactus’s Needle model, which turns text into function calls. Together, they can support voice commands such as turning off a light. The video advises adding function-related terms as keywords to reduce recognition mismatches.

Projects highlighted

Early community projects include Windows and terminal dictation tools, a port of an existing voice-input tool, a Home Assistant integration, a Docker-based transcription server with an API compatible with OpenAI-style transcription clients, and a creator app that generates word timestamps locally.

One example runs Whistle on a modified Echo Show for local smart-home voice control. The user initially saw poor results with an accented voice: 70 of 170 commands were recognized. After training a small additional model to map Whistle’s output to intended commands, the result improved to 164 of 170. The presenter uses this example to argue that Whistle works especially well when the expected vocabulary is constrained.

Main speakers and sources

• Main speaker: The video’s narrator from ungatedAI, who explains the model, presents setup guidance, and evaluates its trade-offs.
• Other sources cited: Cactus Compute’s benchmark results, a third-party German comparison and browser performance test, and community developers’ projects and user-reported results.