Video summary

Your OS Changes Everything for Local AI

Main summary

Key takeaways

Product Review

Product reviewed

GEEKOM A9 Max (mini PC) — updated from last year’s A9 Max. It uses an AMD Ryzen AI 9 HX 470 (Strix Point APU), targeted at local AI / LLM workloads.

Key specs mentioned

  • CPU/APU: Ryzen AI 9 HX 470 (Strix Point), 12 Zen 5 cores
  • iGPU: Radeon 890M
  • NPU: 50 TOPS
  • Memory (test unit): 32GB DDR5-5600
  • Storage: 2TB SSD
  • Ports (noted as unusually full for its size):
    • Dual USB4
    • Dual HDMI 2.1
    • Dual 2.5GbE
    • 5× USB-A
    • Audio jack
    • Side SD card reader
  • Packaging/configuration issue called out: the tested configuration behaves like single-channel RAM because it has only one SODIMM installed (i.e., single stick).

Main features for local AI use (OS/software angle)

The video focuses heavily on which OS yields the best LLM performance on the same hardware, comparing:

  • Windows
  • Windows Subsystem for Linux (WSL)
  • Bare-metal Linux

It also tests multiple inference stacks:

  • Ollama
  • llama.cpp using the Vulkan backend (referred to as Vulkan / “Vulcan”), with notes about GPU offload behavior
  • AMD “lemonade server” (hybrid behavior described as NPU prefill + GPU decode)

On Linux, it compares:

  • radv (open-source driver)
  • ROCm

Benchmarks / performance findings (core takeaways)

1) Windows: Ollama doesn’t properly use the iGPU by default

  • Windows + Ollama (default): around 100% CPU, with GPU mostly idle.
  • Even when Vulkan is enabled, Ollama still appears to fall back to CPU.
  • Better approach: use llama.cpp with Vulkan:
    • GPU utilization rises to about ~100%
    • Gains over Ollama-CPU are roughly:
      • ~10% on smaller models
      • shrinking as models grow (e.g., ~14% at 1.5B, down to ~10% at 14B)

Windows verdict (within the video):

  • If you want GPU usage on Windows, skip default Ollama and use llama.cpp + Vulkan.
  • Even then, Windows still didn’t reach expected peak performance due to a separate bottleneck: memory bandwidth.

2) Windows + AMD “lemonade server”

  • Hybrid approach: NPU handles prefill, GPU handles decode.
  • Reported decode performance is roughly tied with the llama.cpp Vulkan path:
    • about ~1% apart at 3B

3) WSL: GPU works, but throughput is lower than native Windows/Linux

  • WSL + Ollama: still CPU execution.
  • WSL + Vulkan via llama.cpp: requires extra setup:
    • additional Mesa/Vulkan components (e.g., kisak-mesa)
    • driver setup involving dznICD.json / Mesa Vulkan drivers
  • After setup:
    • GPU becomes visible in WSL
    • GPU utilization around ~83%
  • Throughput penalty vs Windows-native GPU setup:
    • WSL is about ~16% slower (roughly 1/6 throughput loss)

4) Bare-metal Linux: best performance for long-context “prefill”

  • Linux Ollama: reported ~99% GPU utilization
  • llama.cpp on Linux:
    • radv and ROCm decode were within a few percent of each other
  • Biggest OS win is for long prompt prefill (the “paste your entire repo” / RAG / agent workflow):
    • Linux (radv) prefill ~3× faster than Windows in the 14B scenario

Linux verdict (within the video):

  • Long-context prefill: Linux radv wins by about ~3×
  • Decode: fairly similar across OSes once GPU is engaged

The dominant bottleneck: memory bandwidth (single-channel vs dual-channel)

The video argues that OS/backend differences largely collapse into whether RAM bandwidth is the limiting factor.

Diagnosis

  • The tested GEEKOM A9 Max config appears to be single-channel (only one DDR5 SODIMM installed).
  • The HX 470 design can use both memory channels, but with one stick installed it runs single-channel bandwidth (about half).

Fix recommended

  • Install a matched pair (use two SODIMMs / dual-channel).
  • If the unit ships single-channel, budget additional cost to add the second module (suggested as roughly $80 more, framed as ~$300 in today’s money).

Performance after upgrading to dual-channel (64GB total)

  • Decode performance roughly doubled
  • Reported multipliers:
    • Windows + Vulkan: 2.13× faster
    • Linux radv: 2× faster
    • Ollama + ROCm: 1.86× faster

Conclusion: GPU compute wasn’t the limiter—the RAM bus was.

“Which OS wins?” (decode results)

The creator provides consolidated “chip capability” numbers for decode across OSes (as presented in the chart narrative):

  • 9× on the 14B model
  • 17× on the 7B model
  • 37× on the 3B model

They also state:

  • Windows and Linux are pretty much tied for decode
  • WSL is a bit lower, but has the advantage of mixing Windows + Linux toolchains

Comparison with a similar product / prior test

  • Compared against an earlier sanity-check on a Beelink SER 10 (similar class).
  • Results were “very close.”
  • The earlier test indicates:
    • Beelink already includes dual-channel RAM
    • GEEKOM does not in the tested configuration

The creator suggests GEEKOM should include two chips (dual-channel) by default.

Model/version concerns addressed

  • The creator anticipates complaints about “old models,” saying the test is relative, not absolute across every revision.
  • They mention validating with Gemma 4 (12B) as a “newer” model to support the conclusions (including decode/memory behavior), while noting it’s a different architecture.

Pros (unique points mentioned)

  • Excellent port selection for the size (USB4, HDMI 2.1, dual 2.5GbE, many USB-A ports, SD reader).
  • Strong Linux advantage for long-context prefill (RAG/agents/coding assist workflow), about ~3× faster with Linux + radv.
  • GPU utilization can reach near-maximum depending on software stack (e.g., llama.cpp Vulkan on Windows, Ollama on Linux).
  • WSL can use GPU acceleration after extra Mesa/Vulkan driver setup, enabling a combined Windows+Linux dev workflow.
  • NPU present (50 TOPS) and AMD lemonade server suggests potential hybrid inference (NPU prefill + GPU decode).

Cons (unique points mentioned)

  • Windows + Ollama default: fails to use the AMD iGPU properly (falls back to CPU).
  • Even with correct software, performance is capped if RAM is single-channel (one SODIMM installed).
  • WSL throughput overhead: about ~1/6 slower vs native in the test context.
  • Decode is similar across OSes, so switching OS alone won’t fix decode bottlenecks.
  • WSL requires manual setup (e.g., kisak-mesa / Mesa Vulkan drivers) to make the iGPU actually work.

Overall recommendation (verdict)

  • Best for long-context workflows (prefill / RAG / agent typing large prompts):
    • Bare-metal Linux with radv
    • clear winner at roughly ~3× faster in the long prefill test
  • Best “decode” performance:
    • less about OS, more about dual-channel RAM
    • if the unit ships single-channel, upgrade to dual-channel: it roughly doubles decode
  • For Windows:
    • use llama.cpp + Vulkan
    • avoid default Ollama to ensure GPU usage

Final take: Buy the GEEKOM A9 Max if you’ll ensure dual-channel RAM; then choose Linux (radv) for maximum long-context gains. On Windows, avoid the default Ollama path.

Speakers/views

Only one main speaker is present in the subtitles; no separate contributors’ viewpoints were introduced.

Original video