Liquid LFM3 Is INSANE – Tiny Local AI That ANYONE Can Run!

Summary

Bijan Bowen reviews Liquid AI’s LFM3 family, a group of small models designed to run locally on modest computers and edge devices. The lineup includes a 1.7B-parameter text model, vision and audio models, an “omni” model for video and other inputs, and a decision model that the video does not test.

The text model has a stated 256K-token context window, and a base version is available for fine-tuning. The weights are available, but the license is proprietary rather than MIT or Apache.

Hands-on Tests

• Text model and agent tasks: The 1.7B model was tested in Q8 quantization on a Surface Laptop using a custom web interface and Hermes agent. It responded quickly and could find files, edit a simple browser-style interface, and complete a web-search task that produced a referenced, Wikipedia-style report. Its coding was limited: the interface was simple and had some spelling and functional shortcomings. However, the agent’s ability to use tools and navigate files stood out for a model of its size.
• Vision: The model identified a dog breed, interpreted some of a Subaru head-gasket meme, and read text from a laptop image. It also converted a photo of a vehicle fuse-and-relay diagram into a table, getting much of the information right but missing some entries. Bowen notes that the test image was imperfect, making this a practical rather than idealized OCR test.
• Audio: With voice-activity detection added to the demo, the audio model supported hands-free, turn-by-turn offline conversation. It responded quickly but did not support natural interruption in this setup and made mistakes, including inventing contact information and misinterpreting part of a request. The demo’s voice interface was custom-built, so these limitations may reflect the setup as well as the model.
• Omni and video: The model quickly answered a question about a short promotional video, but its response included an apparent misreading. It did not appear to use the video’s audio in this test; Bowen suggests this could have been an issue with the demo pipeline rather than the model’s intended capabilities.
• Edge-device test: The 1.7B model ran in 4-bit quantization on an Arduino Uno Q with a Qualcomm DragonWing chip and 4 GB of memory. Bowen reports roughly three tokens per second. He presents this as a demonstration of local AI running on inexpensive hardware, while acknowledging that it was not especially fast.

Overall Assessment

The main appeal is accessibility. Bowen says a computer from roughly the past decade should be able to run at least one of the models locally, potentially even on CPU, though speed will vary by hardware.

The tests show useful basic capabilities across text, vision, audio, and video, but also reveal clear limitations in accuracy and task complexity. The models’ small size makes them practical for experimentation and on-device use, while the proprietary license is an important consideration.

Reviews and Tests Covered

1. 1.7B text-model and Hermes-agent tasks
2. Image understanding and OCR
3. Hands-free offline voice conversation
4. Video understanding with the omni model
5. 4-bit inference on an Arduino Uno Q edge device

Main Speaker and Sources

• Main speaker/reviewer: Bijan Bowen, from the Bijan Bowen YouTube channel
• Products and tools discussed: Liquid AI’s LFM3 models, Hugging Face model releases, Hermes agent, and Arduino Uno Q