Video summary
Fable 5.1 vs GPT-6 Astra: Tested on 74,000 Lines of Code
Main summary
Key takeaways
Summary
Duke Pan compares Fable 5.1, GPT-6 Astra, and GPT-5.6 Sol on a complex business-design question involving three repositories and about 74,000 lines of code. Rather than testing simple website or game-generation tasks, he chose an ambiguous problem: whether a proposed semantic layer was a sound way to organize company data.
A semantic layer maps business concepts—such as “sales information”—to the underlying tables and data. This helps AI find and interpret information across many tables while accounting for differences in business definitions and levels of detail. For example, revenue might be recorded per employee account or per business unit. The model needs to understand those different levels, or it may produce incorrect joins, groupings, or totals.
Test results
Model Cost Time Main strengths Fable 5.1 About $8 15–16 minutes Most detailed analysis and documentation GPT-6 Astra About $2 3 minutes Best balance of speed, cost, and useful findings GPT-5.6 Sol About $1 4 minutes Cheapest, but less detailedThe models reached broadly similar design recommendations, which Pan judged technically sound. Their main differences were the depth and type of supporting analysis:
- Astra identified a significant omission: the design did not adequately model different levels of data granularity. This could lead to incorrect analysis when filtering or aggregating data. It also considered edge cases and referenced an external Snowflake article.
- Fable 5.1 produced the most complete deliverable, including a comparison of four design options, a roughly 250-line design document, and a migration plan. It reviewed more materials than the other models, but was also the slowest and most expensive.
- GPT-5.6 Sol found a similar overall direction but provided less supporting detail. It did not identify the same technical gap or produce comparable documentation.
Pan notes that Fable and Astra can serve different purposes: Astra is suitable for fast design analysis, while Fable is better when a thorough written design and migration plan are needed. His overall pick for this test is Astra, based on its combination of speed, cost, and important technical findings.
Cost and workflow takeaways
Pan cautions that similar API price rates do not necessarily mean similar real-world costs; agent behavior, token use, and cached reads can affect what a task actually costs. He argues that expensive models may be most useful as designers or orchestrators, while cheaper models handle much of the implementation work.
He gives DeepSeek V4 Flash and GPT-5.6 Luna as examples of lower-cost implementation options, while saying he plans to test Astra further.
Reviews, guides, and comparisons covered
- A hands-on comparison of three models on a real codebase and design problem.
- A review of their cost, speed, source coverage, and quality of findings.
- Practical guidance on choosing a model for design analysis versus detailed documentation.
- A suggested workflow: reserve a capable model for orchestration and design, and delegate implementation to cheaper models.
- A comparison cited from Nate Herk on Fable 5.1 versus Fable 5, which Pan uses to illustrate how implementation-model choices can dominate total agent costs.
Main speaker: Duke Pan Other sources mentioned: Nate Herk, a Snowflake article on semantic layers, and Kimi K3, which Pan used to compare the models’ findings.
Rate this summary
Your feedback will help improve summaries.
Improve this summary
Reprocess with a stronger model when the summary feels incomplete or inaccurate.
Translate summary in another language
Ask questions to this video
Chat for follow-up questions, clarifications, and source-backed answers.