에이전트가 올린 5만 5천 줄 PR — Notion 엔지니어가 말한 "진짜 병목은 리뷰가 아니다"

Core Thesis

The subtitles argue that even as AI agents improve automated correctness checking and reduce how much humans must manually review, human understanding still matters. Review and learning change the person, and accumulated understanding enables better creative work in future iterations.

---

Key Technological / Product Ideas and Methods

Verification Loop + Decreasing Human Involvement

• Agents can run stronger verification loops, reducing the chances humans are shown incorrect outputs.
• The speaker cautions that people may mistakenly conclude they no longer need to understand anything.
• Understanding is framed as what makes you an active creative participant, not just a passive checker.

---

“Explain Diff” Skill for PR / Code Change Understanding

• A daily-use system that converts a code diff into an explanatory document.
• Output formats can include HTML or Markdown, often designed to be placed in Notion for collaboration, commenting, and review.
• Example: a pull request that changes a game camera from top-down to isometric.

Output Structure (Guiding Principles)

1. Teach the system first Start with the engine, coordinate system, and relevant subsystems before describing the change.

2. Intuition before details Begin from the commit’s goal/essence rather than dumping code immediately.

3. Interactive figures when useful For example, draggable simulations (e.g., rocks) that illustrate coordinate and Z-layer changes.

   • Notion recently launched support for HTML blocks in Notion pages, enabling embedded interactive simulations. 4. “Literate code diffs” Prose and explanation are arranged in the right order, describing what’s happening before each file—so diffs read more like an explanation than a raw artifact.

Readability Beyond the IDE

• The explainer can be read away from the IDE (even “at a coffee shop”), framed like reading a textbook about the PR.

---

Quiz-Based Comprehension Check (Spaced Repetition / Retrieval Practice)

• Addresses the problem that “reading is hard”: people may believe they understand without actually understanding.
• Inspired by research work by Anima Anand and Michael Nielsen, using interactive spaced repetition quizzes to force real comprehension and memory.
• Implementation described:
  • The code explainer includes a 5-question, medium-difficulty quiz at the bottom.
  • Rule: the speaker doesn’t send PRs for team review unless they can pass the quiz about what the agent wrote.
• Presented as a “speed regulator”—to prevent the team from optimizing only for speed-of-correctness instead of speed-of-understanding.

---

Microworlds / “Mathland” for Learning Code

• Inspired by Seymour Papert’s idea of learning in a microworld (often called “Mathland”): learners change their understanding by doing/experiencing concepts.

• Examples:

  • Building a Prolog interpreter debugger UI:

    • Lets you scrub through a step-by-step timeline
    • Visualizes internal state
    • Supports comments on the timeline Used for both bug fixing and developing “peripheral vision” that agent-only debugging wouldn’t provide.

  • Migrating a personal website as a game-like, step-by-step interactive process:

    • Old vs. new
    • Clickable “next”
    • Shows commands and file-tree changes This creates iterative understanding without manual pain.

---

Big-Picture Conclusion

• AI should be used not only to ship code, but to create ephemeral UIs, dynamic simulations, and debuggers/playgrounds that help humans understand.
• The speaker claims this aligns with historical roots of the field (notably Alan Kay): computers should empower learners to modify and learn through interaction, not merely automate tasks.
• Overall optimism: with the right tools and mindset, understanding can improve—“better than ever,” rather than diminishing.

---

Main Speakers / Sources Mentioned

• Notion engineer (primary speaker; referenced as the “Notion engineer”)
• Anima Anand
• Michael Nielsen
• Seymour Papert
• Alan Kay
• Claude (assistant tool used in examples)