So I've been testing Gemini 4...

Summary

The video reviews Gemini 4 “Argan” after an eight-month gap since Google’s last Pro-class model release. The reviewer tests its pricing, front-end design, 3D generation, and coding performance, comparing it with models including Opus 5.5, GPT-6.1 Soul, Fable 5.1, and Muse Wonderful.

Pricing, Availability, and Benchmarks

• Google launched the model for developers and enterprises through its API and AI Studio, and for eligible subscribers through its apps and Antigravity.
• The introductory API price is $2 per million input tokens and $10 per million output tokens, scheduled to rise to $4/$20 on December 1.
• The reviewer argues that the introductory price can be misleading: the model uses many tokens and, in his testing, had unusually poor cache-hit rates, which can increase actual costs.
• He notes that Google’s benchmark claims—including a high ranking on the VL Index—do not reflect his results on real coding tasks. He also cautions that benchmark success depends heavily on what a benchmark measures.

Product Capabilities

Front-End Design

Gemini produced some polished-looking marketing-page designs, especially when used with Anthropic’s design skill. The reviewer found it less compelling for iterative product-interface work, where speed and reliable back-and-forth matter.

Vision and Google-Specific Tools

Gemini may be worth testing for complex vision tasks. The reviewer also expects to keep using Google models for YouTube-related tasks, such as retrieving video transcripts, because of Google’s API access and data.

3D and Games

In the reviewer’s “Fishslop” game test, Gemini created a playable game loop and reasonable fish animation, and improved some issues after feedback. However, the first version had broken mechanics and visual problems. Later changes still included game-design mistakes, such as a power-up that expired before it could be useful.

Skateboard-Trick Benchmark

Gemini achieved a perfect score on the reviewer’s private benchmark and appeared reasonably efficient on that task. He stresses that this narrow result should not be generalized to other work.

Coding Tests and Reliability

The reviewer’s strongest criticism comes from codebase audits and hands-on coding tasks:

• In an audit of a TypeScript-to-Rust compiler project, Gemini made 36 claims, 24 of which the reviewer judged false, earning a substantially lower score than the comparison models.
• In other audits, the reviewer again found many inaccurate findings, including a project review where 56 of 78 findings were wrong. He says these results made him distrust Gemini’s code reviews and advice.
• In a task involving a database-connection issue, Gemini identified part of the problem but proposed an oversized, incomplete change. Its pull request was poorly formatted, failed checks, and did not account for existing proposed fixes. The reviewer says another model later identified a much smaller, correct change.
• The reviewer also reports slow runs, API errors, a failed Antigravity connection that ended a long session, and integration problems with Google’s coding-agent interface.

The video distinguishes between hallucination benchmarks and practical usefulness: a model may avoid answering uncertain questions yet still provide incorrect or unhelpful analysis when asked to work on a real codebase.

The reviewer’s overall conclusion is strongly negative for coding. He says he would limit his use of Gemini to Google-specific data tasks and possibly some initial marketing-page design work, while continuing to use other models for software development.

Guide and Sponsor Segment

• The video is primarily a hands-on review rather than a step-by-step tutorial. It demonstrates model-testing workflows, including coding audits, design-skill comparisons, and a 3D game-generation task.
• A sponsor segment describes WorkOS as an authentication and enterprise sign-in platform, and introduces a standard intended to let AI agents register for apps on a user’s behalf.

Main Speakers and Sources

• Theo, the video’s host and reviewer, provides the tests and conclusions.
• Google is the source of the model’s launch information, product claims, and cited benchmark and customer statements.
• Artificial Analysis and other model-comparison charts are discussed as external benchmarks.
• The reviewer also cites his own benchmarks and codebases, an unnamed Google employee’s anecdotal comment, and a separate model’s review of Gemini’s pull request.