Proving AI Wrong: Running GPT-OSS 120B on $75 Tesla M10 GPUs & DDR3 - 7.7tps!

Summary

The video tests whether large language models can run locally on older, inexpensive server hardware. The creator pairs three NVIDIA Tesla M10 cards—each described as having 32 GB of RAM—with a Supermicro X9SCMF motherboard and 16 GB of DDR3 server memory.

When asked whether the setup could run GPT-OSS 120B, Google’s AI assistant, Gemini, repeatedly said it was “mathematically and physically impossible.” The creator then demonstrates the system running the model, contradicting Gemini’s assessment.

The reported performance was about 7.7 tokens per second for generation and nearly 35 tokens per second for prompt evaluation. The creator describes the results as usable, though not especially fast. These figures are the creator’s reported results; the subtitles do not provide independent verification or explain the software configuration in detail.

Guide or tutorial

• The video points viewers to the creator’s webpage or README for details about the hardware setup and results.
• A more detailed walkthrough of assembling the system is mentioned as a possible future video, but is not included here.

Main speaker and sources

• Main speaker: Gary Davenport, the system’s builder and presenter.
• Other source discussed: Google Gemini, whose claim that the hardware could not run the model is tested and challenged.