Video summary

I Battle Tested Sakana Fugu's Fable Killer

Main summary

Key takeaways

Technology

Summary of the Video (Technology/Product Concepts + Review/Test Results)

What “Sakana Fugu Ultra” is

  • Sakana Fugu (by sakana.ai, Japanese) is positioned as a multi-model “multi-agent system” delivered as one API.
  • A key claim: it doesn’t behave like a single “new best” LLM. Instead, it uses a small manager/conductor model that:
    1. Breaks down the user task
    2. Delegates subtasks to specialized frontier models (examples mentioned: Opus, GPT, Gemini, etc.)
    3. Combines outputs into a final response (via another LLM/combiner step)
  • The concept is described as a “mixture of experts” via orchestration, where different models are used for different parts (e.g., writing vs coding/bugfixes vs research/facts).

Demonstration: YouTube dashboard + AI analysis

  • The speaker tests Fugu Ultra inside Claude Code using cloud code, running a prompt to generate a dashboard.
  • Claimed capabilities shown in the resulting dashboard:
    • Refreshes live data
    • Displays stats like audience pulse, distribution/performance, median/outliers
    • Highlights what is working, rising, and underperforming
    • Provides recommendations and AI analysis
    • Can analyze videos with specific metrics per video
    • Can review audience and comments, plus strategy
  • The prompt is described as a one-shot “slash goal prompt” to Fugu Ultra, returning the dashboard after ~an hour (speaker calls it impressive).

How it compares to similar orchestration APIs

  • The speaker compares Fugu’s approach to OpenRouter Fusion:
    • Fusion: sends the prompt to multiple models simultaneously and uses a judge/merger to combine.
    • Fugu: breaks tasks into parts and delegates dynamically (a manager decides who does what), then combines results.
  • General takeaway: using multiple perspectives/models can improve quality, but costs and speed usually increase.

Setup/tut guidance mentioned

  • The speaker says they can provide a markdown file and instructions (via a “free school community”) to help people run Fugu Ultra inside Claude Code by:
    • Adding their API key
    • Using the shared markdown template/file
    • Deploying/running through Claude Code
  • They also mention including more technical details in those markdown/HTML resources, including how routing/context handling differs from simply changing an endpoint.

Main Review/Test: “Battle Testing” Fugu Ultra vs Opus 4.8

Test design

  • The speaker runs 38 tasks across four waves (examples mentioned: puzzles, traps, specs, heavy algorithms).
  • Codex is used to create the tests so they’re consistent/objective-ish.
  • Both systems are given the same prompts/inputs, and Codex grades outputs.
  • They emphasize this is not a fair “single-model intelligence” comparison because Fugu isn’t just another LLM—it’s an orchestrator/manager wrapping other models.

Outcome (quality)

  • Result summary:
    • 36/38 tasks ended in a tie
    • Opus won in 2 tasks (the speaker mentions “Opus 1,” but also notes Opus 4.8 is part of Fugu’s model pool)
  • Overall quality feeling:
    • The speaker says Fable noticeably felt better than Opus 4.8
    • But Fugu Ultra did not noticeably feel better than Opus 4.8 for their knowledge-work use case

Outcome (speed + cost)

  • Speed
    • Fugu total runtime: 357 minutes
    • Opus total runtime: 80 minutes
    • Fugu is ~4.5x slower overall
    • Even “easy” tasks: Opus sometimes answered in seconds, while Fugu took minutes for similar prompts
  • Cost
    • Opus: about $10
    • Fugu: about $50
    • ~5x more expensive
  • Conclusion:
    • If quality is roughly the same (ties), the speaker doesn’t see enough value to justify waiting longer and paying more for their workflow.

When the Speaker Thinks Fugu Might Be Useful

  • Potential value when you have complex product/dev workflows with many components/roles, such as:
    • A GPT reviewer + Claude-style planner/IDE workflow via an orchestrated “all-in-one” API
  • Their personal use case is knowledge work, not heavy product building or large-team codebases—so orchestration overhead doesn’t pay off for them yet.

Honest Takeaway / Recommendation

  • The speaker believes orchestration is a future direction, and that unit economics + routing/efficiency (choosing the cheapest model that doesn’t sacrifice quality) will become an important skill.
  • However, for their needs: Fugu Ultra isn’t worth switching to yet.

Main Speakers/Sources

  • Speaker: The YouTube narrator/reviewer (no specific name provided in the subtitles).
  • Primary source/product referenced: Sakana AI Labs / sakana.ai — “Fugu” / “Fugu Ultra” (multi-model orchestration API).
  • Comparators referenced: Opus 4.8, Claude Code, OpenRouter Fusion API, and Codex.

Original video