Can 10 Year-old $5,700 GPU Beat a New $430 GPU? | Tesla P100 Local AI Review

Product reviewed

Nvidia Tesla P100 (data center accelerator GPU, launched 2016) — tested as a budget option for running local AI models and compared mainly against RTX 5060 Ti.

---

Key features highlighted (Tesla P100)

• 16GB 2nd-gen HBM (HBM2), described as still relevant for memory-bound workloads compared to mid-range GPUs.
• No display outputs (requires a second GPU for monitor output).
• EPS 12V power connector (not 8-pin) → needs a power adapter (not always included).
• Comes with a passive heat sink only → requires an external blower/fan; 250W TDP means you can’t run it “cheaply.”

Software/runtime considerations

• Latest CUDA no longer supports P100, so older CUDA is required.
• In LM Studio, runtime settings must be adjusted to use older CUDA.
• For newer workflows (e.g., image generation), PyTorch/CUDA downgrades and additional flags are needed.

---

Setup / user experience notes (clunkiness vs convenience)

The video describes the P100 as not plug-and-play:

• Must use another GPU for display output.
• Requires an EPS 12V adapter (verify before purchase).
• Needs an external fan solution; the reviewer used a 3D-printed/P100-specific setup and manually controlled fan speed.
• Requires older CUDA (and additional tweaks for certain apps).
• Overall: high friction and complexity compared with the consumer RTX card.

---

Benchmarks and results (P100 vs RTX 5060 Ti)

1) Dense LLM (Qwen 3.6-27B, quant IQ3_XXS)

• P100
  • Prompt processing (PP): 127 tokens/sec
  • Token generation (TG): 10.37 tokens/sec
• RTX 5060 Ti
  • Prompt processing (PP): 388 tokens/sec (~3× faster)
  • Token generation (TG): 10.07 tokens/sec (P100 slightly faster)

Interpretation given:

• P100 struggles in compute-bound prompt processing (dated architecture + lack of tensor cores).
• P100 remains competitive in memory-bound token generation thanks to HBM2.

---

2) Mixture of Experts (Qwen 3.6-35B MoE, quant IQ3_XXS)

• P100
  • PP: 295 tokens/sec
  • TG: 35 tokens/sec
• RTX 5060 Ti
  • PP: 589 tokens/sec (~2× faster; gap shrinks vs dense model)
  • TG: 52 tokens/sec (P100 loses this time)

Interpretation given:

• P100 can still be usable, but the advantage becomes less consistent.

---

3) BF16-type model test (P100 lacks native BF16 support)

• Reviewer runs: Quant 3.54B at BF16 precision
• RTX 5060 Ti: “absolutely destroyed” P100 in PP, >4× faster
• P100: TG only ~10% slower in generation

Interpretation given:

• BF16 performance penalty is severe due to no BF16 hardware, but the reviewer notes real-world use often relies on quantized GGUF instead of BF16, reducing practical impact.

---

4) Image generation (ComfyUI workflows)

• P100: needs PyTorch/CUDA downgrade + flags (noted as tricky)
• RTX 5060 Ti: much faster
  • ~4× faster with Flux client B
  • ~2.5× faster with Z image turbo

Interpretation given:

• Compute-heavy generative image tasks are where the P100 becomes especially painful.

---

Price/value claims

• Original P100 pricing referenced: $5,700 launch; market price up to ~$7,000 at the time.
• Current market: as low as ~$80 on eBay.
• Claimed value strategy:
  • At $80, you can buy six P100s to pool VRAM at roughly the same cost as one RTX 5060 Ti.

---

Pros (as stated)

• Strong performance for memory-bound tasks (HBM2 helps in token generation).
• Budget value: very low acquisition price; potentially good VRAM-per-dollar.
• Can be viable for local quantized LLMs if that’s your primary goal.

---

Cons (as stated)

• Dated architecture and lack of tensor cores → very poor for compute-bound tasks (especially prompt processing).
• Image generation is painfully slow on P100.
• Clunky setup: no video output, special power connector, requires external cooling hardware.
• Software support limitations: requires older CUDA; additional compatibility work for some tools.
• High power consumption (250W TDP).

---

Comparisons made

• Repeated direct comparisons against RTX 5060 Ti across multiple workloads:
  • Dense LLM: P100 slightly better in TG, much worse in PP.
  • MoE LLM: P100 closer in PP; loses in TG.
  • BF16: P100 >4× slower in PP; generation only slightly behind.
  • Image generation: RTX 5060 Ti 2.5×–4× faster.

---

Overall verdict / recommendation (concise)

• Recommendation: Buy the Tesla P100 only if you’re specifically targeting cheap local, quantized LLM workloads and you can tolerate setup complexity and slow performance in compute-heavy tasks.
• If you want a fast, plug-and-play experience, the video strongly favors the RTX 5060 Ti.

---

Unique points mentioned (consolidated list)

• P100 is a 2016 data center GPU.
• 16GB 2nd-gen HBM (HBM2) remains useful vs some mid-range GPUs.
• P100 can be found for ~$80 on eBay (vs much higher historical pricing).
• No display output → needs another GPU for monitor.
• Uses EPS 12V instead of 8-pin → power adapter may be required.
• Comes with passive cooling → requires external blower/fan; 250W TDP needs proper cooling control.
• Requires older CUDA (newer CUDA drops support).
• Dense LLM: PP ~3× slower; TG roughly about equal/slightly better.
• MoE: PP gap narrows (~2× faster on RTX); TG wins for RTX.
• BF16: RTX >4× faster in PP; TG only ~10% slower on P100.
• Image generation: P100 requires more software setup; RTX is ~2.5×–4× faster.
• Final framing: P100 is a budget VRAM / quant LLM option, but not for plug-and-play speed (prompting + images especially slow).

---

Speakers

• The subtitles indicate one primary speaker/reviewer conducting the setup and benchmarks (no distinct multiple-speaker viewpoints were labeled).