Meta Muse Glimmer 30B tested - 16GB Local LLM setup

Product reviewed

Meta Muse Glimmer 30B — an open, dense (non–mixture-of-experts) local LLM aimed at “always-on local agents,” tested in a local 16GB VRAM setup using GGUF quantized to 4-bit (UDQ4K XL).

Test setup & constraints (user experience impact)

• GPU/VRAM: RTX 2000 Ada with 16GB VRAM
• System RAM: 32GB DDR4
• Key limitation: 30B doesn’t fit entirely in 16GB VRAM, so it relies on system memory (“offload”), reducing speed.
• Max context tested: 128K (also stated as the model’s maximum in these tests).

Main features mentioned

• Dense 30B model (explicitly not MoE).
• Designed for local agents / tool-calling workflows (tested via Agency and MCP challenges).
• Supports very large context (up to 128K tested).

Benchmarks & numerical results

Performance (speed)

• Prefill (context ingest) at 130K context:
  • ~13 tokens/s down to ~4.5 tokens/s for larger prefill
• Decode (output generation):
  • ~6.2 tokens/s consistently (criticized as “not great”)
• Comparison: Quen 27B (dense) reportedly runs around 10–11 tokens/s on the same tester.

Memory / long-context retrieval

• Tested across depths (0%, 25%, 50%, 75%, 100%) across 15 runs total.
• Overall pass rate: 87%
• Failure pattern:
  • At 75% depth: data found 1 out of 3 → 33% pass rate
  • Other depths: data found every time
• Output quality:
  • Generally clean, but some failures were noisy / included extra “thinking/padding.”

Agency / tool sandbox

• 47 out of 49 scenarios passed (~ballpark typical for models of this size).
• Comparison: Similar performance to Quen 27B and 35B (often ~48/49).

OpenAI HumanEval (handwritten Python)

• Pass rate total: 71%
• Pass rate on answered questions: 84%
• Reason for overall failures: the model ran out of context on 26 questions (given 8,000 tokens per question; after 4,000 tokens used, a prompt injection occurs).
• Comparison: Described as standard / on par with dense Quen models around similar size (e.g., 27B/35B).

Coding & agent-style task results (hands-on)

Canban (front-end web dev)

• 128K context, configured per Meta guidance.
• Result: one-shot success
  • Produced full project structure.
  • UI worked: drag cards, toast notifications, add/rename columns & cards, and search behavior mostly fine.
• Minor issues:
  • UI not perfect, but functional.
  • Reordering nuance: it seemed you had to drag to the correct target area; column/drag behavior required correct interaction.
• Overall: “very impressed”—rarely do other models pass this test in one shot.

Sand physics simulator

• Delivered a working interactive HTML-style artifact.
  • Initially didn’t write a file as expected; reviewer notes the prompt wasn’t explicit about saving HTML to disk.
• Once corrected via prompting/interpretation, it supported:
  • placing sand, water, wool (brush size changes), acid, and an eraser
• Overall: strong single-pass functional performance.

Dungeon crawler (algorithmic generation + raycasting)

• Issues encountered:
  • First render: dungeon connectivity/raycasting not right.
  • Subsequent iterations:
    • One fix for dungeon broke player spawn.
    • Another prompt fixed player spawn and allowed movement.
• Final state:
  • Dungeon generation improved: rooms connected, fog of war worked, ray casting looked solid
  • Player could move and explore
• Critically: issues were identified and fixed quickly without “looping.”

MCP tool-calling tasks (Blender + Godot)

Blender assets (marble + gate)

• Generated required assets and exported them.
• Result:
  • Gate + marble present; camera render worked.
  • Files exported correctly (blend file + exports folder).
• Overall: solid for multi-asset creation tasks.

Godot game (platforming + orbs + collect mechanic)

• Claimed completion but early playability issues:
  • Mouse look/control needed refresh/restart (temporary inability to regain mouse control)
  • W/S controls reversed initially
  • Collecting an orb caused crash/death, preventing completion
• After the model fixes:
  • Movement/sprinting/jumping/collision worked
  • Still: orb collection crashes the game, stopping further progress
• Reviewer takeaway: despite the lingering orb bug, the delivery was better than many models and likely fixable with additional context/iterations.

Pros (unique points)

• Strong coding performance, especially Canban passing in one shot with correct functionality.
• Good long-context memory behavior overall (87%), with mostly clean outputs.
• Reliable tool/agent execution:
  • Agency benchmark high (47/49)
  • MCP Blender task completed without mid-run help
• Issue diagnosis + concise repair:
  • Dungeon crawler problems fixed in a small number of prompt iterations
• Usable local experience on limited hardware (4-bit quantization + offloading enabled running)

Cons (unique points)

• Speed/throughput weaker than comparable dense models:
  • Decode ~6.2 tokens/s vs Quen 27B’s 10–11 tokens/s
• Memory retrieval dip at exactly 75% depth:
  • 1/3 success at 75%
• HumanEval overall pass rate limited by context injection behavior:
  • 26 unanswered questions due to running out of context
• MCP Godot still had a critical gameplay bug:
  • Collecting an orb crashed/dead-ended gameplay
• Some deliverable expectations weren’t met due to prompt ambiguity:
  • Sand/dungeon file-writing expectations required reviewer prompt adjustment

Comparisons made in the video

• Quen (dense):
  • Decode speed: 10–11 tokens/s (Quen 27B) vs ~6.2 tokens/s (Muse Glimmer)
  • Agency benchmark: similar ballpark (~47–48/49)
  • HumanEval: described as similar/standard to dense Quen models around size
• Gemma:
  • Mentioned in the initial benchmark comparison list (Gemma 4 31B), but no deep hands-on results were described.
• Planned future head-to-head:
  • Direct comparison against Quen 27B
  • Later comparisons against Quen 27B / Quen 3.8B
  • With matching quantization (Quantization Q4K XL) to mirror test conditions

Overall verdict / recommendation

Recommended if you want a capable local dense model for agentic coding and tool use—it performed very well in Canban, Sand, Dungeon crawler, and Blender MCP, often finishing without intervention.

Not ideal if you’re extremely sensitive to speed (decode noticeably slower than Quen 27B) or if your use case requires perfect MCP game logic in one run (the Godot orb bug remained unresolved).

Unique points by reviewer “speaker” (single speaker)

All points come from Luke (Luke’s Dev Lab); no separate speakers were present in the provided subtitles.