Video summary

Qwen 3.8 27B Quantizations Q1 - Q8 compared

Main summary

Key takeaways

Product Review

Product reviewed

  • Qwen 3.8 (27B) / Qwen 3.8 27B quantizations (EnSoft model tested): comparisons across IQ1M, Q2 KXL, Q3 KXL, Q4 KXL, Q5 KXL, Q6 KXL, Q8 KXL.

Testing approach & tasks

Each selected “best-of” quantization was run through multiple tests:

  1. Canban (web-based task)
  2. Blender (MCP challenge): create “Starfall lantern” with lighting/textures; evaluated by delivered Blender result + screenshots when provided
  3. Godot (MCP challenge): build a 3D platformer with moving platforms; evaluated by gameplay functionality

Reasoning X high was used for all quantizations to give them the best chance.

Key results (by quantization)

Q8 KXL

Canban

  • 84.8% context used
  • UI/interaction issue: an edit-card window popped up immediately and couldn’t be dismissed; required one extra intervention/prompt to fix.
  • Otherwise produced a “pretty good” and feature-complete project; expected “faded preview” behavior.

Blender (lantern)

  • 54.9% context used, no compaction
  • Best visual quality: good textures, convincing glass transparency, lighting; overall strong asset.

Godot (platformer)

  • 49.8% context used after compaction
  • Game mechanics largely working and reviewable.
  • Issues noted: Z-fighting on platforms.
  • Interrupt used because state was “good enough” to review.

Overall take: highest quality visuals/functionality, but not flawless; some interaction glitches (Canban) and visual artifacts (Godot).

Q6 KXL

Canban

  • 80.8% context used
  • Errors: couldn’t move card/column initially; errors occurred, then model fixed them after being given the errors.
  • UI similar to Q8; “one error along the way” mentioned.

Blender

  • 55.3% context used, no compaction
  • Delivered a good lantern, but not as good as Q8 (less impressive textures; still functional).

Godot

  • Required 2 compactions, 92% context used (more costly)
  • Inverted look up/down initially; fixed after prompt.
  • Issues: Z-fighting; a panel blocking view in one place.
  • Game still playable to end.

Overall take: solid quality, but less efficient and more issues than Q5/Q4/Q8 in some areas.

Q5 KXL

Canban

  • 84.1% context used
  • No errors during the run
  • Functional UI; minor UI weirdness: a blue filter banner at top (suspected filter banner; user unsure).

Blender

  • 34.2% context used (very efficient)
  • Screenshots missing initially (model said it looked great, then screenshots weren’t shown); later images were provided.
  • Glass looked strangely angled (possibly design/stylistic but likely wrong).
  • Still delivered images/results.

Godot

  • 76.9% context used, 1 compaction
  • Multiple control issues corrected via prompts:
    • inverted movement axes (up/down and forward/back)
    • initially couldn’t collect orbs; later appeared functional
  • Stopping point: model uncertain about jump; reviewer ended before final full validation.
  • Final state: mechanics worked (orbs/key collection, hazard boxes, respawn), but lighting/performance weaker.

Overall take: best blend of reliability on Canban and decent creative output; artifacts/uncertainty remain in harder Godot scenarios.

Q4 KXL

Canban

  • 86.8% context used
  • One-shot success
  • UI/functionality good; minor issue: icon in search field overlaid on placeholder.

Blender

  • 81.2% context used
  • One-shot, no interaction
  • Lantern looks less textured than Q8; inside glow not visible.
  • Overall acceptable and “pretty good.”

Godot

  • 78% context used, 1 compaction
  • Required interruptions when model’s in-context testing was wrong.
  • Issues fixed via prompts:
    • reversed movement controls early
    • not locked to facing direction
    • reversed backward/forward later
  • Remaining gameplay defect: moving platform teleports at end; animation loops not correct.

Overall take: often the “sweet spot”: reliable and balanced, with manageable imperfections.

Q3 KXL

Canban

  • 87.2% context used
  • One-shot
  • Feature-complete after interaction; some null value observed on card creation until entering card and adding fields (then functional).
  • Reviewer states result is “quite nice.”

Blender

  • 88.8% context used
  • Required one extra prompt
  • No screenshots delivered (initially).
  • After compaction, 23.3% context used (but it used tokens earlier).
  • Issues: hook on top incorrect, but overall looks better than Q4 (reviewer opinion).

Godot

  • Failed: repeatedly crashed the program; after this, opening the project crashes instantly with warning.

Overall take: good on Canban and Blender, but not usable for Godot.

Q2 KXL

Canban

  • 68.2% context used after compaction
  • One-shot with no intervention
  • Fully functional, no errors.
  • Strong surprise: reviewer expected loops/errors at lower quantization.

Blender

  • Initially 47.2% context used, no screenshots; after requesting screenshots, 52.2% context used
  • Glass transparency not visible (less impressive), but:
    • “quick”
    • fits well on GPUs
    • “pretty good for Q2”

Godot

  • No-go:
    • got stuck in a loop (reviewer tried to escape repeatedly; couldn’t).

Overall take: highly viable on Canban and Blender, but fails on Godot due to looping.

Q1 (IQ1M) KXL

Canban

  • Explicitly unable: enters a loop over a plan endlessly; can’t be recovered.

Blender

  • Also fails: tool usage errors (bpy.data delete call), then loops; doesn’t complete the task.

Godot

  • Instantly goes into looping behavior after connecting to tools; no progress.

Overall take: not recommended; “no-go” across tasks tested.

Pros (unique points mentioned)

  • Q2 can be surprisingly viable for non-complicated web-based tasks (Canban) with one-shot success and no errors.
  • Higher quantization (Q5/Q6/Q8) generally improves output quality on creative/visual tasks.
  • Q4 described as a reliably balanced “sweet spot” for quality vs speed on complex work.
  • Q8 and Q4 perform well on Blender asset creation (best lantern look quality especially Q8).
  • Efficiency: Q5 showed low context usage on Blender (34.2%) and generally stable Canban runs.

Cons (unique points mentioned)

  • Q8: Canban interaction glitch (dismissable edit card window issue); Godot has Z-fighting.
  • Q6: more computation cost (high token usage, multiple compactions on Godot); some UI/logic issues requiring error prompts.
  • Q5: UI banner oddity on Canban; Blender had glass panel angle issues; Godot had uncertainty about jumping and some performance/lighting limitations.
  • Q4: Godot moving platform defects (teleporting at end, incorrect animation loops); minor Canban search icon overlay.
  • Q3: Godot crashes repeatedly; Blender needs extra prompt and has a wrong hook element.
  • Q2: works well on Canban/Blender but loops on Godot (complete failure).
  • Q1: heavy quantization leads to endless loops on all major tasks tested.

Comparisons & key takeaways from the reviewer

  • Quality vs quantization level is task-dependent:
    • Web/Canban (simpler): Q2 is surprisingly usable; Q1 is useless.
    • Creative/visual (Blender): quality increases toward Q5/Q6/Q8, with Q4 as a good balance.
    • Game building (Godot, harder mechanics): lower quantizations (Q1/Q2/Q3) fail, while Q4/Q5/Q6/Q8 can produce playable results (with varying defects).
  • For speed/fit: reviewer notes Q2 can fit on a 16GB GPU and is efficient enough for practical use in less complex tasks.

Quantization ranking implied (by outcome)

  • Best overall quality (in tested scenarios): Q8 (especially Blender)
  • Best balance: Q4
  • Promising practical middle: Q5
  • Usable but more costly / more issues: Q6
  • Mixed/unstable: Q3 (good on Canban/Blender, fails on Godot)
  • Surprisingly viable but limited: Q2 (good on Canban/Blender; loops on Godot)
  • Not usable: Q1

Verdict / recommendation

  • Non-complicated web-based tasks (like Canban): Q2 KXL is viable and surprisingly reliable (avoid Q1).
  • Creative quality (Blender-style assets): choose Q5–Q8, with Q4 as the “sweet spot” if you want decent quality without the highest resource use.
  • Harder interactive mechanics (Godot platformer): stick to Q4+ (Q4/Q5/Q6/Q8); lower quantizations often fail/loop/crash.

Speakers / views

  • Single primary speaker (Luke’s Dev Lab): all observations, testing outcomes, and conclusions are from one reviewer.

Original video