Which is the best AI HARNESS ? Ship benchmark with codex, jcode, pi, opencode, dsh

Tech goal / setup

• The video compares AI “harnesses/agents” by generating the same web-based 3D ship simulator using the DeepSeek Flash 4 model.
• A custom prompt is reused across harnesses:
  • Create a relaxing 3D ship simulator for the browser
  • Sun rises/sets with alternating day and night
  • Clicking the sea should create waves

How the test is run

• Each harness requires significant work: about ~50 minutes to ~1+ hour per iteration, largely due to heavy tool calling / build steps.
• The author runs multiple iterations to check determinism.
• Results are not deterministic—different runs can produce notably different outputs.
• The author mentions a safe way to run tests: “sandbox” (be-nix) to avoid breaking the local system.

Harnesses compared (and what was observed)

1. Pi Coding Agent (pi)

• Runs a local web server (e.g., localhost:8000).
• Generates a ship simulation with sun/day-night animation, but:
  • No proper interacting waves (waves/particles don’t interact with the ship well).
  • The sun appears as a “square” (visual oddity).

2. Codeex

• Requires different project execution steps (find/run dev server; uses build/dev tooling).
• Produces a different boat and visuals compared to pi.
• Day/night works with sunrise/sunset and moon animation.
• Waves are present but:
  • They are not very wave-like and don’t interact strongly with the ship.

3. DSH (DeepSeek harness)

• Uses compiled outputs and relies on a setup that can use a real browser (Chromium/Firefox mentioned generally).
• Key strengths:
  • Real waves
  • Waves interact with the boat
• Tradeoffs:
  • Inverted controls for moving.
  • Time/acceleration limitations: the demo requires waiting and can’t be sped up.
  • Sunset reflection looks good, but sun visibility is inconsistent (sometimes not clearly visible).

4. Jcode

• Tested multiple iterations:
  • One iteration uses Firefox + an extension/bridge to observe browser development feedback.
  • Another iteration likely uses a different browser setup.
• Observations across iterations:
  • Voyager-style ship motion appears in early runs.
  • Waves may start in the wrong location (center of the boat instead of the click location) in some iterations.
  • Some iterations include sounds (the author muted them later).
• Best-performing run (author’s view):
  • Jcode + Firefox integration via bridge gives the most realistic waves and better alignment with the prompt (waves triggered by clicking, with suggested interaction with the ship).

5. OpenCode

• Uses a standard structure and is served locally (e.g., localhost:8080).
• Produces a different boat.
• Waves:
  • Described as quite realistic
  • The animation is considered strong
  • It’s unclear whether waves physically interact with the boat, but the wave animation is arguably the best among some harnesses.
• Sun animation:
  • The sun renders as black, but motion/animation is acceptable.

Overall conclusion / review-style takeaway

• With the same model (DeepSeek Flash 4) and the same prompt, different harnesses—and even repeated runs—produce very different results.
• The author’s opinion:
  • Jcode with Firefox integration is the most realistic overall (best waves/interaction behavior).
  • However, harness performance varies: one may be better at waves, another at sun/moon animation, etc.

Main speakers / sources

• Single primary speaker/creator: the video author (who runs the tests and shares opinions).
• Referenced sources: the tested harness projects—pi coding agent, Codeex, DSH (DeepSeek harness), Jcode, OpenCode—and the project sandbox for safe testing (be-nix).