Video Summary - Qwable 3.6 27B tested

Tech summary (Qwable / Qwen fine-tune, local 16GB LLM test)

Model under test: Qwable
- A Qwen model fine-tuned on “Fable 5” style reasoning/instruction data.
- Subtitles mention training on “Unsloth’s Qwen 3.6 27B” with cleaned Fable 5 style reasoning and instruction data.
Quantization / runtime: 4-bit quantization in GGUF format so it can be run with llama.cpp.
Test goal: Run the same set of browser-based coding challenges as prior 27B comparisons to benchmark/compare output quality.

Inference system: 16GB VRAM and 32GB DDR4
- Explicit note: the 27B model cannot fully fit in 16GB VRAM, so offloading is required.
Development workflow:
- VS Code with a client extension to connect to llama.cpp
- A GitHub link was referenced for prompt files and model configs

Initial behavior: Model followed the prompt and produced code/checklists.
Major failure mode:
- Got stuck in a verification loop due to context length explosion
- Attempted ~100K context while limited to 64K
Required intervention: Manual kill/restart of the instance after the loop.
After fixes: Game ran, but with additional issues:
- Background initially blank, later improved to show labels (e.g., trees/meadows)
- Crash handling: On crash it resets to start, and pressing R to restart didn’t work; required a page reload
Overall sentiment:
- Worse than the “Qwable/Quipus” 27B comparison
- Also worse than the base model (per the speaker’s opinion)
- One positive: car orientation was sometimes handled better

Output: Produced the index file and appeared relatively one-shot
Functionality observed:
- Materials: sand and water work
- Sand/water interaction: described as somewhat buggy/glitchy
- Building/walls: wall placement restricted—can’t build through existing materials
  - Speaker notes this wasn’t present in earlier tests, but doesn’t strongly penalize it
- Acid: degrades materials over time (not instant); progresses in gradual “steps”
- Eraser, brush size, clear: all worked
Overall sentiment: Generally good, with acid behavior specifically called out as interesting/different

Expected tasks: Ray casting and procedural generation, using algorithm options
Major failure mode: Browser became stuck loading; DevTools wouldn’t load
Observed looping behavior:
- Repeatedly opened the browser
- Tried verification/screenshot-like steps
- Entered loops that caused system lag
Required intervention: Repeated stopping/killing
Prompt adherence issue: Poor adherence—model keeps opening the browser even though guidance said not to use it directly
Conclusion: Author largely abandoned this test as non-progressing and too resource-intensive

Initial run:
- Took about 5.5 hours
- Score: 53%
- Pass rate: 88
- Author suspicion: harness likely not broken; model probably spent too much “thinking budget” without producing final answers
Second run (after tweaking):
- Nearly 6 hours
- Many “no answers” again
- For questions it did answer: 90%, but overall it still didn’t answer enough
Final decision: Author chose not to further tinker due to time cost

Not recommended: “Personally, I’m just going to say I wouldn’t bother with this one.”
Reasons cited:
- Weak performance across challenges
- Poor prompt adherence
- Looping behavior
- Heavy compute usage without successful completion
Upcoming: Next comparison planned with Gemma 12B, trained on Fable and Cursor Composer

Speaker: Luke (host of “Luke’s DevLab”)
Primary model referenced: Qwable / Qwen 3.6 27B fine-tuned on Fable 5 (via Unsloth)
Runtime/tools referenced: llama.cpp, GGUF 4-bit quantization, VS Code extension, GitHub-linked prompts/configs (source not otherwise detailed)