Extreme Harness Engineering: 1M LOC, 1B toks/day, 0% human code or review — Ryan Lopopolo, OpenAI

Core idea: “Harness engineering” to remove humans from the SDLC

• Ryan Lopopolo argues that coding models + a “Codex harness” can automate most of the software development lifecycle (SDLC).
• The key is letting the model operate inside an engineered environment (“wiring”), then iterating from:
  • traces
  • tests
  • documentation
  • review feedback
• The approach emphasizes systems thinking:
  • Continuously ask where the agent makes mistakes
  • Identify where human time is being wasted
  • Encode fixes back into the harness so future runs converge.

Quantified internal results (zero human code authoring)

• Internal effort took about ~5 months
• During that period, they report:
  • zero lines of code written by humans
  • reaching ~1M total LOC / ~~1.5k PRs
• Framed as roughly ~10x faster than manual work.
• The work started with earlier Codex tooling (Codex CLI + weaker models) and evolved as model generations changed.

Adapting to model changes: background shells + build time discipline

Model capability shifts required changes to the build system:

• Earlier versions lacked “background shells,” so long tasks relied on blocking scripts.
• With background shells (notably around 5.3), the model became less willing to wait/block, requiring builds to complete quickly.

They enforced a tight loop:

• ~1 minute build cap for the inner loop
  • If builds don’t finish, the system signals failure
  • The workflow decomposes the build graph / retries to keep CI under the limit
• A “gardening” / ratchet mindset prevents build times from drifting upward.

“Humans became the bottleneck” → minimize synchronous review

The system scales because:

• Models are trivially parallelizable (tokens/GPUs)
• Human attention is scarce

So human involvement shifts toward:

• post-merge
• release gating
• No continuous deploy; releases require a human-approved smoke test before promoting to distribution.

Prompting philosophy: encode non-functional requirements in text + tooling

Lopopolo’s critique is that the harness should externalize reliable engineering behavior:

• Docs and tests for rules of the road
• Linting / error messages that teach correct behavior
• Review agents and task breakdown instructions

Key admonition:

• Avoid “garbage prompting” (injecting irrelevant context)
• Instead, ensure prompts provide only what’s useful and necessary for correctness.

Observability as a first-class input to the agent

The harness provides:

• traces
• logs
• metrics

So the model can diagnose and fix issues without humans digging through terminals.

He reframes this as:

Not “humans debugging traces” Instead “agents fixing the system” using traces as feedback.

“Skills” and markdown as cheap scaffolding + shared team knowledge

They “reinvented skills” (skills didn’t exist when they started).

The repo includes agent guidance such as:

• spec.md
• agent.md (short/structured, with a table of contents)
• Skills like:
  • a tech debt tracker
  • a quality score

Tech-debt/quality tracking is implemented as:

• text + small scaffolds
• Used by agents to:
  • review guardrail compliance
  • propose follow-up work
  • keep the system’s “institutional knowledge” updated

Review-agent negotiation and escalation controls

Their review workflow includes guardrails to prevent loops and scope explosion:

• Review agents comment on PRs
• The coding agent must acknowledge/respond to feedback
• Controls include:
  • priority scoring in prompts (e.g., “P2” / “P0” style frameworks)
  • reviewer agents are biased toward merging unless issues are severe
  • coding agents can defer or push back when feedback should become backlog/future work

Version-control / collaboration: work as multi-agent PR flow

He notes that Git can be hostile to multi-agent workflows, but claims it can work with:

• agents
• work trees

They run agent-driven cycles:

• push PRs
• wait for review + CI
• fix flakes
• merge upstream
• handle merge queues / resolve conflicts as needed

Human intervention is mostly minimized to:

• releases / smoke tests
• occasional termination

Deploying “everything” through the harness

The system is described as capable of handling many responsibilities in parallel, including:

• product code + tests
• CI configuration
• release tooling
• internal dev tools
• documentation
• eval harness
• responding to review comments
• scripts that manage the repository itself
• production dashboard definition files (e.g., Grafana JSON)

Introducing “Symphony” (agent orchestration via Elixir/BEAM)

Symphony is positioned as removing humans from terminal-driven context switching as PR volume grows.

Core mechanism:

• Each PR/rework cycle can be trashed and restarted when review escalates (if not mergeable)
  • enabling cheap iteration
  • improving reliability

Origins:

• scaling PRs per engineer
• reducing “tmux pane” context-switching overhead

“Specs” distribution: ghost libraries / reproducible local reconstruction

With Symphony, they generate a spec that encodes enough to reproduce a system locally (even across repos).

Workflow:

1. Write a spec
2. Spawn Codex in tmux to implement the spec
3. Spawn Codex to review differences vs upstream
4. Update spec iteratively until fidelity is high

This is framed as a reusable way to distribute complex knowledge and tooling cheaply.

Broader platform direction: Frontier (enterprise agent governance)

Ryan describes OpenAI Frontier as an enterprise platform for safely deploying agents at scale with:

• an “agent SDK”
• works-by-default harnesses
• governance, observability, safety, and integration with enterprise IAM/security tooling

Additional concepts:

• “safety specs” to prevent exfiltration and enforce enterprise-specific policies
• an internal data agent to make company data ontologies usable for agents (semantic framing like “what is revenue”)

Where automation still struggles (and where humans remain)

Hard remaining gaps:

• Translating end-state product mocks into a playable/working product for net-new ideas (zero-to-one)
• Deep refactors / monolith decomposition

Expectations:

• improvements as models get better at codebase understanding and interface shaping
• harness/scaffold work should focus on giving models the right non-functional requirements

Collaboration tooling need

Lopopolo argues future agent productivity depends on collaboration layers (GitHub/Slack/Linear-style workflows) so agents can coordinate with humans economically and effectively.

---

Main speakers / sources

• Ryan Lopopolo (OpenAI) – main interview subject; discusses harness engineering, Codex CLI/harness, Symphony, and OpenAI Frontier.
• Podcast interviewer (unnamed in subtitles) – asks questions and provides commentary/clarifications.