Video summary
Stop Searching For the Perfect Model
Main summary
Key takeaways
Summary
Theo argues against constantly choosing the “best” AI model for each kind of work or building elaborate model routers. Instead, he recommends using different models together on difficult or high-impact changes. Their distinct training and approaches can uncover issues that one model might miss.
The “Council” workflow
Theo’s Tstack skills include Council, which brings in several models—typically from different model families—to review a change. The models can inspect and run the code, test it, ask the primary coding agent questions, and provide feedback. Reviews are scored from 1 to 10, and the agent is asked to keep improving the work until it reaches a specified threshold, such as 9.5.
Unlike review bots that run only after a pull request is filed, these reviewers can participate during development, while the code is still being changed. Theo also uses the workflow to assess whether a change would meet a particular person’s preferences, not just whether it seems technically sound.
Theo says models from the same family can provide less varied feedback, comparing them to the same reviewer at different times. Different model families, by contrast, have raised useful and sometimes novel concerns.
Reported results and examples
Theo shares results from 144 reviewer scores across his work:
- Reviewers initially scored the work below 7 on 21 occasions; only twice did the final score remain below 7.
- In 128 cases, the final score was 9 or higher.
- In one comparison, Opus’s average score rose from 7.4 to 8.9 after review rounds; Muse’s rose from 7.7 to 9.1, and Soul’s from 7.6 to 9.1.
Reviewers caught issues including:
- Valuable tests that had been deleted during cleanup, and tests that had been restored despite being unable to fail.
- Security problems in T3 Code’s guest-sharing feature, including access to visuals after revocation and race conditions around share access.
- Edge cases in interrupted or failed turns that could leave the interface in the wrong state or override a user’s action.
Theo notes that a model can catch issues when reviewing code it wrote itself, because the reviewer role encourages different behavior.
Skills and practical cautions
Theo describes Full Send, Full Stop, and Soul Loop alongside Council. Full Stop aims to have an agent finish, verify, and prepare a change before asking the developer to review it. Soul Loop uses repeated review-and-revision cycles.
The main goal is to keep the developer out of the agent’s work until it is ready for human attention. Theo says this can make agent-assisted development less distracting and increase the odds that a pull request is ready to merge.
There are several trade-offs:
- Cost: Multi-model reviews can consume many tokens. Theo says they are most practical with subscriptions and advises considering the expense before using the workflow at direct API prices.
- Runaway loops: Review cycles can run too long, generate excessive code changes, and expand the scope. Theo recommends limiting the number of rounds, stopping when progress stalls, and resisting scope creep.
- Excessive delegation: Theo describes a run that spawned hundreds of subagents and ballooned a modest change into a much larger one.
Sponsored product
The video also features Macroscope, which Theo says checks every pull request in the T3 Code repository. Its programmable review agents can be configured in a repository folder to focus on selected file types and code-specific rules, with options for the model and reasoning level. Theo describes it as a customizable, agent-based system for pull-request checks.
Main speakers and sources
- Theo — Presenter describing his coding workflow and reported results.
- AI models used in the Council examples — Opus, Muse, Soul, and Gemini. The subtitles’ model names may be imperfect because they are auto-generated.
- Macroscope — Sponsor and provider of programmable pull-request checks.
- Lauren (“Potato”) and the Pstack skills — Cited as an inspiration for Theo’s Tstack approach.
- T3 Code contributors and community users — Referenced in examples and audience feedback.
Rate this summary
Your feedback will help improve summaries.
Improve this summary
Reprocess with a stronger model when the summary feels incomplete or inaccurate.
Translate summary in another language
Ask questions to this video
Chat for follow-up questions, clarifications, and source-backed answers.