Which is The Best Qwen3.8-27B?

Main technological idea

The video compares multiple fine-tuned/retrained variants of Qwen3.8-27B designed to reduce the amount of “thinking” / “reflection” tokens used during inference. The goal is to improve latency and potentially cost, while attempting to keep accuracy similar.

A key concept is reasoning effort (Qwen3.8 out of the box):

• Settings: low / medium / extra high
• “Extra high” is default after launch in the tested setup.
• Changing reasoning effort alters the length of reasoning chains, which affects:
  • time to first answer
  • overall generation cost
• On-device/locally, decoding speed makes reasoning tokens roughly correspond to wall-clock wait time.

The speaker frames the tradeoff as:

Models that “rephrase, double-check, think again” use more tokens → more waiting time → more money.

---

Models discussed (and intended improvements)

1) Thinking Cap (Bottle Cap AI) – optimized for less reasoning at “extra high”

Goal: Reduce the number of thinking tokens without changing the model’s:

• knowledge
• instruction style
• safety
• communication …meaning behavior stays mostly the same, but with shorter reasoning.

Training approach (high level):

• Trained at X-high reasoning effort
• Focused on harder evaluation areas:
  • math
  • logical reasoning
  • long context
  • agent tasks
• Key metric: fewer thinking tokens with only minor accuracy loss

Reported results/claims:

• ~37% fewer tokens across 12 tests
• Accuracy changes slightly: 86.6% → ~85.8%
• Example: GPQA Diamond thinking tokens: 12,800 → 7,300
• In some long-context tests:
  • token savings ~39%
  • accuracy sometimes increases by a few points
• Compared to simply lowering reasoning effort on the base model:
  • Lowering to medium: ~half the reasoning, but ~9 points accuracy drop
  • Thinking Cap (X-high): ~37% cost reduction with ~1 point accuracy loss
• Agent-related downside: agent trajectories show only ~11% reduction in reasoning (still shorter, but less savings than other tasks)

Practical positioning:

• A near drop-in replacement for Qwen’s X-high model for general reasoning tasks.

Licensing:

• License changed vs earlier versions:
  • from Apache 2
  • to a more restrictive license aimed at small businesses/personal use

---

2) Swift 1.5 (Yu Kex AI) – more aggressive reflection reduction; strong RL/distillation

Goal: Reduce “reflection tokens,” avoid specific triggers of unnecessary thinking, and use stronger training (RL/distillation) to improve performance with less reasoning.

Training approach:

• Stage 1 (Swift 1.0):

  • Identify reasoning tokens that seem to provoke unnecessary loops
  • Fine-tune to avoid such tokens (not only penalize length)
  • Examples of reduced behavior:
    • “Wait, let me ask what the user wants…”
    • “Let me rethink…”
  • Incorporated into the Thinking Cap lineage

• Stage 2 (Swift 1.5):

  • Scale up reinforcement learning
  • Use strategy-based distillation
  • Focus on long-term agent planning tasks

Reported results/claims:

• Up to ~58.5% fewer reflection tokens on Qwen3.8-27B
• Coding benchmark improvements:
  • Live Code Bench: ~77% → 81.7%
  • Output token count: ~11,200 → 8,400
• Savings claimed to hold across effort levels:
  • At low: slightly higher score than base while using ~29% fewer reflection tokens
• Behavioral claim: the base model may loop / terminate prematurely, while Swift does not (per the speaker’s observations)

Ecosystem/support:

• Multiple deployment options:
  • Llama.cpp GGUF
  • small low-power variants
  • MLX
  • NV and AMD builds
• Provides a free research API for experimenting without a key

Licensing:

• Proprietary, free up to $1M revenue, then contact them.

---

3) Qwen Pi (Apache 2) – specialized for a minimal coding-agent environment (Pi)

Goal: Instead of reducing reasoning at X-high, the Pi approach targets efficiency at low/medium effort by training for a specific environment.

What Pi is:

• An open-source coding agent with a minimal toolset:
  • read/write/edit/bash
• Very small system prompt (< ~1000 tokens mentioned)

How Qwen Pi is trained:

• Guided retraining on real Pi sessions, but only from successful runs
• Manual verification that code actually works (emphasis on correctness checking)
• Reinforcement learning with GRPO, plus rewards for reasoning efficiency
• Reward targets prioritize low/medium settings; X-high focuses more on correct execution
• Checkpoints chosen by agent performance, not just training loss

Positioning/expected benefits:

• Makes reasoning more efficient in that specific agent environment
• Claim: Pi can match base model X-high performance on a terminal benchmark using ~41% fewer source tokens
• Concept shift:
  • Thinking Cap/Swift shorten X-high reasoning
  • Pi increases “intelligence at the lower effort” for agent coding workflows

Limitations:

• Narrow use case: evidence gathered primarily inside Pi
• For general logic tasks (e.g., GPQA), the base model may still win on average

Licensing:

• Mentioned as the only one of the three with Apache 2

---

Experimental/testing described in the video

The speaker runs comparative demos on similar hardware:

• Dell Pro Max tower with RTX Pro 6000
• No special optimization by the speaker

Method:

• Generate queries designed to elicit many reflections/reasoning tokens
• Save outputs with timestamps for side-by-side comparison across:
  • base Qwen 3.8 27B
  • Qwen Pi
  • Thinking Cap
  • Swift 1.5

Qualitative outcomes:

• For some categories (queries that trigger lots of reasoning):
  • Thinking Cap often finishes fastest / most concisely
• Swift 1.5 often runs fast and can rank high, especially on certain tasks (including coding-like ones)
• Qwen Pi is efficient for coding-agent style tasks, but may be slower on non-agent reasoning settings
• The speaker emphasizes variability:

  Results vary by question type, so testing all variants can be worthwhile.

---

Guides/tutorials or reviews angle

The coverage is framed like a review/comparison breakdown, including:

• explanation of the underlying reasoning effort mechanism
• a “family tree” / lineage view of how the variants relate
• review of each model’s:
  • training strategy
  • reported benchmarks
  • token-efficiency claims
  • licensing
  • deployment options
• an on-device comparative demo across multiple task prompts

---

Main speakers/sources (end)

• Main speaker/source: the video reviewer/author (unnamed in subtitles; speaker references prior videos and personal testing)
• Models/labs referenced:
  • Bottle Cap AI (Thinking Cap)
  • Yu Kex AI (Swift 1.5)
  • Pi (open-source coding agent) and its ecosystem
  • OpenAI mentioned as an industry reference for reducing token-based reasoning chains