Video summary

What It Really Costs to Run DeepSeek R1 at Home | George Hotz

Main summary

Key takeaways

Technology

Summary

George Hotz estimates what it would take to run DeepSeek R1 locally on a home-built, CPU-only system. His central point is that memory bandwidth—not just memory capacity or processor compute—limits token speed.

Memory bandwidth and model type

In a simplified example, a system with 1 TB of RAM and 1 TB/s of bandwidth could read all the weights of a dense model once per second, yielding about one token per second. A mixture-of-experts model may be faster because it uses only some of its weights for each token: accessing roughly 10% could mean roughly 10 tokens per second. Quantization can reduce the memory required.

Why not stream weights to a GPU?

Hotz argues that transferring weights over PCIe would bottleneck performance. He cites about 64 GB/s for PCIe 5 and says that moving weights from CPU RAM through the CPU to the GPU would be too slow to make GPU computation worthwhile.

GPU memory costs

Hotz discusses why GPUs may not simply offer much more VRAM: memory adds substantial cost, and he questions whether market demand is sufficient. As rough examples, he mentions an 8 GB memory cost of $27 and estimates GPU RAM at around $6 per GB.

Estimated R1 performance and build cost

Using R1’s 37 billion active parameters and theoretical bandwidth from two fourth-generation EPYC CPUs, Hotz estimates a potential speed near 25 tokens per second, while noting that real-world performance would be lower.

He estimates that a 768 GB system with fast DDR5 could cost about $10,000–$15,000, with RAM alone costing roughly $4,000. He also mentions about $2,000 for around 3.5 tokens per second as a rough estimate.

Cheaper builds

Used DDR4 server memory might cost around $1 per GB, compared with roughly $5 per GB for DDR5. Hotz suggests a system costing about $3,000–$4,000 could reach roughly five tokens per second, while emphasizing that cheaper memory means lower speed. NVMe drives are also mentioned as an option for very slow operation.

Practicality

Hotz questions the value of building such a machine when local speeds may be modest and the API is inexpensive. He also says Tinygrad might build a small CPU-only system, but presents this as a possibility rather than a released product.

Reviews, guides, or tutorials

No formal product review or step-by-step build guide is provided. The video is a rough technical and cost analysis of possible local R1 hardware.

Main speaker and source

George Hotz, speaking in a stream clip published by George Hotz Clips.

Rate this summary

Your feedback will help improve summaries.

Improve this summary

Reprocess with a stronger model when the summary feels incomplete or inaccurate.

Pro

Translate summary in another language

Pro

Ask questions to this video

Chat for follow-up questions, clarifications, and source-backed answers.

Coming soon

Share this summary

Original video