Local AI Cluster At Home with 5-Device (125B AI model)

Reported Performance Claim

A Reddit user claims to run the 125-billion-parameter Qwen 3.8 Flash Next model across two PCs, a MacBook, and two iPhones at about 40 tokens per second, even with long prompts. The claim—including how the devices are connected and how the model is divided—has not been independently verified.

How the Approach Could Work

A model processes tokens through layers in sequence, so each device can hold a contiguous section of layers instead of the entire model. The model’s sparse design reduces the computation used per token, but not the storage needed for the assigned weights. Devices pass a small hidden state between them; the weights remain in each device’s memory.

This works like an assembly line: each device handles its assigned work and passes the result onward. However, each token must wait at every stage, and a failed device or connection can interrupt the whole pipeline.

Documented Example: Backburner

Developer StayLameBro describes using a MacBook for the early layers of a smaller Qwen model and an iPhone 17 Pro Max for later layers over USB-C. This is not a demonstration of Flash Next running on iPhones.

Backburner’s README reports about 5 GB of weights on the phone and roughly 2.6 MB transferred over the cable per token chunk.

Limitations and Practical Guidance

Backburner’s README says the setup does not speed up answer generation below roughly 64,000 tokens of context; its benefit is in reading long prompts. It handles one request at a time, and a device failure can cause about 60 seconds of downtime.

The developer’s measurements indicate that a regular iPhone’s slower USB connection is unsuitable; a Pro model is needed for this setup.

Bottom Line

The layer-splitting approach has a documented example, but the claimed 125-billion-parameter, five-device setup and its reported speed remain unverified. The video recommends trying Backburner only if you have compatible hardware and long prompts, and waiting for independent reproduction before relying on the Reddit performance figures.

Main Speakers and Sources

• The video’s narrator
• An unnamed Reddit user making the 125B performance claim
• StayLameBro, the Backburner developer, whose README provides the documented setup and measurements