All Code Is Slop (Except Mine) — Lauren Tan, SpaceXSI

Summary

Lauren Tan argues that AI coding agents can produce large amounts of useful code, but only when the development environment makes correct behavior the default. The key is earned trust, not blind trust: improve the codebase, constrain agents with deterministic checks, and verify their work.

Main Ideas and Technical Practices

• Treat the codebase as an agent’s input. Agents imitate existing patterns, including technical debt, workarounds, and anti-patterns. Keeping the codebase clean helps prevent bad practices from spreading.
• Prefer enforceable rules to written instructions. When an agent makes a mistake, Tan recommends turning the lesson into a lint rule, type or dependency check, better API, or architectural constraint—not merely adding a note to an AGENTS.md file or other documentation.
• Use architecture to prevent common errors. The Dune client framework, described as similar to Next.js for Electron apps, encodes constraints and lessons into its design. Examples include:
  • Disallowing React useEffect in product code except in designated parts of the framework, because it is easy to misuse.
  • Separating Electron renderer and main-process code into distinct directories and prohibiting imports across the boundary to avoid performance problems.
  • Using lint rules to steer agents toward fixing root causes rather than applying superficial workarounds.
• Be cautious with comments as instructions. Tan says comments can become stale or give agents a rationale for preserving a bad fix. Her team reportedly bans most comments and prefers actionable lint errors.
• Build a strong verification process. Code review increasingly focuses on evidence such as videos, screenshots, diagrams, and formal verification artifacts. Runtime verification lets agents run the application and use debugging tools such as the Chrome DevTools Protocol. A “feature map” provides a shared, user-oriented vocabulary for navigating app functionality; reusable CLI tools can make these workflows more efficient and testable.
• Apply formal and automated testing where useful.
  • TLA+ is used to model core user flows, including notifications across devices and platforms.
  • Lean is used to prove properties of algorithms, including a virtualization library intended to avoid flicker and jitter.
  • Property-based interface testing with Bombadil has reportedly found UI bugs.
  • A state-chart tool in the Pstack plugin helps visualize user-facing state machines.

Tan stresses that formal verification is an additional layer, not a cure-all. She also observes that agents lower the cost of writing and maintaining tests, making more extensive verification practical.

Reported Results and Automation Example

Tan says the Grokbot team submitted 11,000 pull requests to production in one month, while the rate of reverted or canceled changes remained around 0.5%. She also reports faster product interactions and a stable client release about once a day. In the talk’s opening, she separately says she sent 3,000 pull requests to production the previous month.

As an example of a more automated workflow, each release can trigger a Cursor project that fuzzes the application by using it like a real user, opens pull requests for discovered bugs, and automatically merges fixes. Tan attributes this level of autonomy to the quality and safeguards of the underlying codebase.

Practical Recommendations

1. Improve the codebase before building a large-scale agent “software factory.”
2. Add linting and architectural constraints to stop recurring anti-patterns.
3. Clean up existing technical debt that agents might otherwise copy.
4. Invest heavily in runtime, automated, and—where appropriate—formal verification.
5. Automate broader workflows only after the underlying code-writing and review process is reliable.

Main speaker/source: Lauren Tan, also known as “Potato,” speaking about the Grokbot team at SpaceXSI.