I Tested Qwen 3.8 in 4 Coding Harnesses—One Clear Winner

Summary

The video compares how four coding harnesses affect the speed, reliability, and output quality of the same local model: Qwen 3.8 Flash Next, running with a newer Unsloth quantization than in the creator’s earlier tests.

Test Setup and Approach

• Tests ran on a system with an RTX 5090, 128 GB of DDR5 memory, and a Threadripper CPU.
• The creator used 16 experts in the Unsloth setup, prioritizing accuracy over maximum token speed. They reported around 22 tokens per second with the same quantization on an RTX 3080 Ti system.
• Each harness received the same general prompt to create a small, island-themed fishing game. The creator also revisited two earlier projects: Neon Salvage and an alien island.
• The comparison was hands-on rather than tightly controlled: setup, prompting, iteration, and verification varied between harnesses. The creator also noted that the newer quantization may have contributed to improvements in some rematches.

Harness Results

OpenCode

OpenCode was the clear winner in the creator’s tests.

• It completed the fishing game in about 18 minutes, producing a polished result from the initial prompt without iteration.
• The creator found its UI the clearest and most usable. It featured animated scenery, fish shadows, and a catch-timing indicator.
• The creator observed OpenCode opening and testing the game in its built-in browser, then returning to the code to fix issues without being asked. They suggested that this ability to inspect and verify its work may explain its strong results.
• OpenCode also produced improved versions of Neon Salvage and the alien-island scene. The creator praised their animation, effects, and overall presentation.
• The creator plans to use OpenCode going forward, while acknowledging that the newer quantization could also explain some of the improvement.

DeepSeek Harness

• It produced a functional fishing game, but the creator needed roughly 12 iterations to improve its detail and readability.
• The game included fish rarity and catch-history features, along with animated scenery. However, the character and some environmental elements were difficult to interpret.
• The creator reported roughly 159 tokens per second and a run time of around 40 minutes.
• They suspected the harness might be less effective with a non-DeepSeek model, but did not establish that conclusively.

Pi

• The creator stopped the run after more than an hour because it became stuck in repeated verification loops.
• They said they had to intervene many times and estimated making more than 50–60 prompts.
• The result included some appealing animation and gameplay details, but the UI was criticized as poor.
• The creator cautioned that Pi may not have been configured properly, but did not recommend it based on this experience. They also found its command-line workflow and setup burdensome.

Cline

• Cline required multiple iterations and was slower than OpenCode and DeepSeek Harness in the creator’s measurements. The recorded attempt took about 32 minutes, with a follow-up after an interruption.
• Its game had a retro look and included sound effects—the only one of the four to do so, even though the prompt did not request sound.
• The creator liked the visual style but found the UI oversized and noted that the game repeatedly cast in nearly the same location.

Main Takeaway

The creator strongly favored OpenCode, especially for users seeking a mostly autonomous workflow. They attributed its advantage partly to its built-in tools for viewing and testing generated projects, while emphasizing that the comparison did not isolate the harness from other factors, such as model quantization and differing follow-up prompts.

The creator plans to test again after a future Qwen release and mentioned a possible future experiment involving changes to the model’s weights.

The video also announced plans to showcase viewers’ software and repositories in future episodes, subject to a safety review.

Main Speakers and Sources

• Otis, the video’s host and tester, from Otis’s AI Lab.
• Viewer comments and suggestions, including a recommendation to revisit the Strata setup and update the Unsloth quantization.
• Tested tools and model: OpenCode, DeepSeek Harness, Pi, Cline, and Qwen 3.8 Flash Next.