Video summary

How We Solved Agent Building — Andrew Qu, Vercel

Main summary

Key takeaways

Technology

Motivation: from web apps to “agentic” applications

  • Andrew (Vercel chief of software) describes a shift in what users want to build:
    • not just pages/web apps
    • but AI agents that handle real business workflows (e.g., analytics, design, product management)

Problem: building agents across many LLM providers is painful

  • Vercel’s AI/DK (as described in the talk) is positioned as an abstraction layer so you don’t need to rewrite hundreds of lines of provider-specific code.
  • Goal: swap one line of code while keeping a consistent model interface across providers.

Supporting agent capabilities (infrastructure-level tooling)

The team built tools for:

  • Model fallbacks
  • Secure code execution
  • Better pricing for “inactive/waiting” time
  • Durability and resumability (agent runs can persist and continue)

The “agent building” journey (D0 → better architecture → file-system unlock → skills → Eve)

1) Early approach: mega-prompt + manual SQL testing

For an experiment aimed at Vercel’s data team, the first approach was effectively:

  • Put a Snowflake schema dump into a prompt
  • Ask a question
  • Let the model generate SQL
  • Manually run it to validate correctness

Conclusion: models may not reliably produce correct SQL yet, but improved context/guardrails could help.

2) Multi-agent pipeline (“D0”): planning → SQL → execution → reporting

They mapped the phases a data scientist does into separate agent workloads:

  • question handling / query planning
  • SQL generation
  • execution
  • reporting/analysis (including visualization and narrative)

Implementation detail:

  • Multiple agents with narrow system prompts and scoped tools
    • e.g., the planning agent uses only tools like schema reading/search until it produces what the next stage needs

Benefit: moved from “copy/paste SQL manually” to end-to-end loops. Limitation: they “hit walls” with this architecture.

3) Single “mega agent” with internal state/memory

They redesigned toward one agent that:

  • contains “mega context”
  • manages its own state across phases (planning/building/executing/reporting)
  • handles errors by reflecting and exploring what went wrong (e.g., retry after tool/execution failures)

Mechanism described:

  • one AI call, up to max steps (~100), with internal state tracking

Outcome:

  • worked “well” in controlled tests
  • but when released to trusted users, results were unexpectedly bad
  • manual scenario mapping didn’t scale

4) Key unlock from Claude/Opus + “file system agents”

A major improvement came when stronger models (e.g., “Claude Code” + “Opus 4.5” mentioned) were released.

The breakthrough wasn’t just the model—it was that the agent had a file system toolset:

  • minimal tools like list/read/write files and run bash
  • the agent could explore emergent behavior and discover solutions by manipulating local artifacts

Inference: “just a file system” (plus a sandbox) can dramatically improve agent reliability.

5) Rebuilding “cloud-code-like” capability in a Vercel sandbox

They recreated the capability as a sandbox:

  • dump necessary context (e.g., semantic layer) into the sandbox
  • provide tools for bash and file operations
  • add Vercel-specific tools as needed

Result:

  • eval score doubled

Claimed launch impact:

  • the blog post states that in the week it launched, it generated ~70% of Vercel.com traffic (as claimed)

Scaling the approach: reusable “skills”

As the system was used, they noticed many user questions shared structural patterns (e.g., aggregation, specific customer/product/billing lookups).

They created a mechanism to distill repeated query shapes into agent skills.

  • Current state (as described):
    • ~100 skills
    • a recurring job converts recent queries into reusable skills

Why skills matter:

  • each agent run starts with some pre-baked contextual knowledge rather than “starting from nothing”

Tooling:

  • “Skillsh” is mentioned as the most popular way to find and run agent skills.

“Next.js for agents”: introducing Eve (agent framework)

The final product direction parallels Next.js:

  • Next.js abstracts filesystem conventions for web apps
  • Eve aims to abstract conventions for agent projects
    • define an agent via folders and conventions

Eve overview

  • Released “two weeks ago” at the time of the talk; referenced as eve.dev
  • Described as “the Next.js for agents”
  • Simple project layout:
    • skills/, tools/, channels/
  • The framework generates an agent runtime from these conventions
  • Supports:
    • custom knowledge
    • custom tools
    • integration with common channels

Architecture components mentioned

  • runtime
  • channels
  • durability
  • isolated execution
  • model calling
  • connections

Open-source and adapters

  • Built to allow plugging in adapters for:
    • Postgres
    • OpenAI Responses API
    • Docker
    • other connectors

Vercel deployment integrations

Eve uses Vercel products for:

  • Workflows/durability
  • Sandbox for secure execution
  • Connect (newly released) for generating short-lived connection tokens

Migration note

  • They rewrote the earlier “D0/Dzero agent” implementation into Eve while developing it.

Validation / early customers

  • Eve was shared with a few beta customers before full release.
  • Example partner: “Aura”
    • used Eve to build a “mini-claw” style agent that tests services (visits websites, installs them, tries to use them)
    • reported:
      • fewer steps
      • better success rates
      • better insights
    • also claimed comparative advantage vs using an “off-the-shelf cloud code” approach

Observability and developer experience

When deploying Eve to Vercel, they claim observability comes out-of-the-box:

  • view agent runs
  • view tool calls
  • view each step
  • estimate costs and suggest optimizations

Getting started:

  • clone a template, deploy easily
  • supports self-hosting if needed

Main message / outcomes

Biggest product/engineering thesis

Best agent performance for their workflow came from combining:

  • strong models
  • sandboxed file-system access (list/read/write/run bash)
  • scoped tools and company/domain context (semantic layer, schema, skills)

Business positioning and claimed impact

Vercel has built many “PMF” agent examples across functions, including:

  • marketing retros
  • sales outreach
  • legal contract redline review
  • data science query assistance

Claimed impact:

  • significant reduction in time spent writing queries
  • improved data team productivity

Main speakers / sources

  • Speaker: Andrew Qu (Chief of Software, Vercel)
  • Referenced external systems/models:
    • Claude Code / “Opus 4.5”
    • Sonnet 4
    • (and their behaviors as observed in the talk)

Original video