Andrej Karpathy: From Vibe Coding to Agentic Engineering w/ Stephanie Zhan

Overview / Key claims

• Andrej Karpathy describes a shift from LLM-assisted coding to “agentic” workflows:
  • Modern models (AlphaCode-adjacent tools, coding agents, etc.) increasingly produce correct multi-step outputs.
  • This reduces the need for human correction.
• He argues LLMs are effectively a new computing paradigm—not just “better software.”

• He frames a progression:

  • Software 1.0: explicit rules/code you write
  • Software 2.0: programming by training models (datasets + objectives + architectures)
  • Software 3.0: programming via prompts/context, where the LLM interpreter executes computation based on what’s provided in the context window

Why he feels “behind” (practical analysis)

• He highlights a rapid transition around December:
  • Earlier agentic code generation often required editing.
  • With latest models, generated code “chunks” increasingly come out correct.
  • He can’t remember the last time he corrected output, which leads to:
    • more trust
    • more autonomous coding (“vibe coding”)
• He suggests many people previously experienced AI as “ChatGPT-adjacent,” but agentic coherent workflows changed things fundamentally by late 2024.

Technological concepts + examples (software 3.0)

1) Installing software as “copy-paste programming” (agent-driven automation)

• Example: when OpenCL installation (subtitled as “Open Claw”) usually requires complex bash scripting.
• Karpathy claims the software 3.0 approach is instead:
  • Copy-paste instructions/text to an agent
  • The agent interprets the environment, performs setup actions, and debugs in the loop
• Key idea: in software 3.0, you don’t script every detail—you supply higher-level instructions, and the agent handles system-specific steps.

2) MenuGen → “prompt an image, get an overlay image” (reducing app/code intermediates)

• He describes MenuGen (an earlier project):
  • Take a photo of a restaurant menu
  • Deploy on Vercel
  • Use OCR/image generation pipelines to recognize titles and generate pictures
• Then he describes a software 3.0 version:
  • Take the same photo
  • Send it to Gemini with a short instruction (subtitles mention “Nano Banana”)
  • Output is an image where items are rendered/overlaid directly in pixels
• Lesson: what used to require a multi-step app pipeline can become one multimodal transformation (image-in → image-out).

3) Knowledge bases as “new outputs,” not just faster code

• He references an LLM knowledge base approach:
  • LLMs can “recompile” documents into wikis/knowledge structures
  • This isn’t even “a program” in the old sense; it’s automated information reformatting
• Key claim: AI enables new kinds of information processing that weren’t previously feasible.

Forecasting future builders (what’s “obvious in hindsight”)

• For the “2026 equivalent” of past eras (web/mobile/SaaS), Karpathy extrapolates from MenuGen:
  • Potentially completely neural interfaces/computers
  • Devices that ingest raw video/audio and render UI using diffusion
• Hardware/software shift:
  • Neural nets become the host process
  • CPUs become co-processors
• He notes the exact trajectory is uncertain, but expects gradual migration rather than a single leap.

Verifiability + “jagged intelligence” (analysis)

Core idea: automation is fastest in verifiable domains

• He discusses verifiability as a reason AI progress appears uneven:
  • Models train in reinforcement learning environments with verification rewards
  • This yields strong performance in math/code-adjacent tasks
  • Performance becomes jagged outside those domains

Examples of jaggedness

• He cites classic “strawberry letters” style issues (implied earlier failures that get patched in newer models).

• A sharper example:

  • State-of-the-art models may answer a navigation question like “walk to a car wash 50m away” rather than drive—despite being able to do extreme code tasks (refactors, vulnerability finding).

• Interpretation:

  • When errors remain, it can mean either:
    • the model is slightly off, or
    • you need tool use / human oversight / in-the-loop correction

Why jaggedness persists (training + lab incentives)

• Models improve where labs:
  • include relevant data/distributions
  • build RL environments that reward verifiable behaviors
• He emphasizes you’re somewhat at the mercy of what labs trained for, so you may need:
  • fine-tuning
  • domain-specific adjustments if your use case is out-of-distribution

Founder/product advice (builders in verifiable domains)

• If you build in verifiable settings, it becomes easier to:
  • use RL environments
  • do fine-tuning effectively
• He implies there are valuable RL-verifiable domains beyond obvious math/code, but doesn’t specify them.
• More broadly, he argues eventually “everything” can be made verifiable to some extent (e.g., using LLM judges), though difficulty varies.

“Vibe coding” vs “agentic engineering”

Difference

• Vibe coding: raises the floor—anyone can generate working software more easily.
• Agentic engineering: preserves the quality/security bar of professional software while increasing speed.

Agentic engineering as a discipline

• Agents are “spiky,” fallible, stochastic—but powerful.
• The engineering task is coordinating agents to go faster without introducing vulnerabilities.

Agent-native workflow expectations

• Hiring:
  • Many hiring processes still rely on old-paradigm puzzles.
  • He suggests hiring should resemble building a large, secure project, then having agents attempt to break it (e.g., Codex-like agents doing security testing).
• Coding skill:
  • AI-native builders differ in how well they use tooling features and how much they invest in their workflow setup.

What humans still must do (taste/judgment + system design)

Even with agents, humans remain responsible for:

• specifying detailed requirements
• ensuring correct identity/linking logic
  • Example: MenuGen credits payment association using email addresses across Stripe vs Google
• “under the hood” correctness can be delegated to agents, but overall design/taste/oversight cannot

He analogizes API details to frameworks (e.g., PyTorch/NumPy/pandas):

• the LLM/agent can handle syntax and call details,
• but engineers still need fundamentals (e.g., tensor storage/view efficiency, avoiding unnecessary copying).

Education / learning value

• He emphasizes: “You can outsource your thinking, but you can’t outsource your understanding.”
• As intelligence gets cheaper, the bottleneck becomes:
  • directing agents effectively
  • forming/maintaining understanding
• He highlights interest in knowledge bases and structured personal wikis to support deeper understanding (including synthetic data generation over fixed corpora).

Main speakers / sources

• Speakers: Andrej Karpathy; Stephanie Zhan (host/interviewer)
• Referenced systems/tools/providers:
  • AlphaCode-adjacent coding agents
  • OpenAI (co-founding mentioned)
  • Tesla Autopilot (historical reference)
  • Gemini
  • “Nano Banana” (image overlay tool mentioned in subtitles)
  • Vercel
  • LLM knowledge base/wiki approach
  • Codex-like agents (subtitled as “codex 5.4x high”)
  • Reinforcement learning/verifiability framed around “frontier labs” and RL training setups