Calibri runs massive 744B parameter AI on laptop without GPU hardware #AI #MachineLearning

Running a Large AI Model on a Laptop

A developer in Italy reportedly ran a 744-billion-parameter AI model on a laptop without a graphics card, using GLM 5.2 and the Calibri project. The video presents this as a challenge to the idea that models of this scale require powerful GPUs.

How It Works

Calibri reportedly keeps roughly 10 GB of model data active in RAM at a time and streams other parts from the hard drive as needed. The video compares this process to streaming a movie.

Setup and Use

The setup involves cloning the project repository and running a setup file. The main storage requirement is downloading about 370 GB of model files.

Once installed, the model can be used locally and offline through a chat interface. The video also mentions a serve command for exposing the model as an OpenAI-compatible endpoint, though the exact command is unclear in the subtitles.

Running inference on-device could help app developers avoid API token charges and keep data from leaving their machines.

Limitations

The video provides a brief setup overview, not a full walkthrough or benchmark comparison. It also claims that GLM 5.2 outperforms paid models, but the subtitles provide no supporting evidence.

Source

The narrator is unidentified. The account is attributed to a developer in Italy and the Calibri project.