I Built One UI for Claude, Codex & Cursor in 3 Hours | Coding by Hand

Summary of technological concepts & what the video builds

• Goal: Recreate a “Supererset / T3.code-like” coding agent orchestrator UI from scratch—initially by hand and in a ~3-hour build—then improve it further to version 2 using the agent itself (self-improving / recursive “inception” loop).
• What the orchestrator does: Provides a single UI to manage multiple coding agent tasks in parallel (e.g., Claude Code / Codeex / Cursor), where each task runs locally on the user’s Mac via provider SDKs.
• Why a backend is required: Agents cannot reliably run “inside the browser” because they need tool access (file read/write, directory access). The project uses a local backend that can spawn/run the agent and stream events back to the UI.

---

Product architecture (T3.code-style vs Conductor-style)

T3.code approach (the one being implemented)

• Frontend (React + TypeScript backend) connects to a local backend (WebSocket/possibly JSON-RPC style).
• Backend:
  • Has access to the full machine (filesystem operations via provider SDK/tool loop).
  • Runs the coding agent loop in-process (Node/TypeScript).
  • Intercepts all agent messages/events (tool calls, file ops, reasoning, final output).
  • Streams those events to the UI and stores them for history.
• Message parsing is the hard part: different agents emit different message shapes, so the hardest implementation work is normalizing/handling agent event streams.

Conductor comparison (from the video)

• Conductor uses the Claude Code UI directly, meaning it doesn’t parse individual message/tool events as deeply.
• T3.code can detect when a conversation ends by inspecting messages; Conductor instead uses optimistic/heuristic checks.

---

Core stack & features implemented

• Frontend: React (then adds Tailwind for UI polish), with:

  • Sidebar for workspaces and sessions
  • Chat pane for messages
  • Rendering special events (e.g., tool calls) in a centered/structured UI
  • Basic handling for “thinking/loader” (Claude-like UX)
  • Markdown rendering improvements (agent helps update renderer)

• Backend: Bun + WebSocket server.

  • Receives JSON messages from the UI
  • Validates inputs via Zod (initially for workspace/session/message creation payloads)
  • Stores and retrieves history in MongoDB
  • Uses WebSocket for streaming UI updates (instead of only storing after completion)

• Database: MongoDB

  • Stores unstructured agent event history as JSON blobs.
  • Rationale: agent messages have varied schemas, which is awkward to model in SQL.
  • Deployment tradeoff:
    • MongoDB can run locally or on cloud (Superset uses cloud).
    • Local storage is possible; cloud storage keeps history across machines.

---

Data model / workflow the video implements

Entities

• Workspace

  • path, name

• Session

  • linked to workspace
  • has conversation array (message/event objects; schema evolves)
  • later tracks provider-specific IDs (e.g., Claude “session id” / entropic session id)

Initial CRUD-like app (agent-free)

• Builds endpoints/handlers for:
  • create workspace
  • create session
  • send user message
• Stores messages in DB and echoes responses (a “ping/pong” stage first, then real UI flows)

WebSocket message protocol

• Uses a typed/shared “commons” package for Zod schemas between frontend and backend:
  • create workspace
  • create session
  • create message (user message)
• Defines outgoing message types such as:
  • workspace created
  • session created
  • message added
  • connection/init response (initial state to populate sidebar)

---

Agent integration (Claude Code agent SDK) and streaming

How it runs the agent

• Uses the provider’s agent SDK (example: Anthropic/Claude agent SDK).
• Backend calls an SDK query function:
  • sends the user’s message as prompt input
  • enables tool access
  • streams intercepted events to:
    • the frontend (via WebSocket)
    • the database (conversation history)

Session handling (important detail)

• Uses the SDK’s session features:
  • continue / resume existing agent sessions to preserve context.
• Tracks “provider session IDs” in DB so later messages can resume the same agent conversation (prevents restarting context every time).

---

Tool call & reasoning event support (UI improvements)

• Initially stores/sends mostly final answers.
• Then extends streaming to include tool-call events (e.g., bash/tool read/write).
• Frontend:
  • Detects events by type (e.g., tool)
  • Renders tool messages in a dedicated UI region (centered, accordion-like idea)
• After improvements, tool calls include enough payload detail to show:
  • tool type (bash/read/write)
  • file paths read/written and command outputs (used later for inspection/visualization)

---

Self-improving “version 2” using the agent itself

• After the main system can answer questions and stream tool calls, the video demonstrates a recursive improvement loop:
  • Ask the agent to modify the app code to:
    • improve tool payload display (turn tool events into richer UI)
    • fix markdown rendering so responses display correctly
    • tweak permissions/approval settings for edit operations (auto-approve bypass mentioned)
• This creates an agent that upgrades its own UI/backend while running (noting downside: front-end can crash due to live code edits).

---

Reviews / guides / tutorials mentioned

Primarily a tutorial/build log.

• “Code by hand” implementation of an orchestrator UI and backend.
• Provides implementation patterns:
  • local WebSocket backend + streaming to React UI
  • shared Zod types via a commons package
  • MongoDB schema-light history storage
  • using provider agent SDKs to run tool loops in-process
  • resuming provider sessions using session IDs
• MongoDB sponsor link is referenced, but no explicit product review beyond the usage recommendation.

---

Key “next steps / TODOs” listed by the speaker

1. Add model selector (Claude/Codeex/OpenAI/etc.) in the UI.
2. Add provider selector (support multiple agent backends).
3. Add SDK integration for other providers:
   • Codeex (check SDK / agent start flow)
   • Devon (mentions ACP / agent context communication server)
   • Cursor (use its agent SDK approach)
4. Additional product features:
   • usage metrics
   • better “worktrees” integration (though Claude/others may already handle it)
   • plugins at super-set/app level (e.g., shared skills per project)
5. More robust UI: improved message renderer, better loaders, session switching reliability, richer tool accordions.

---

Main speakers / sources

• Main speaker: The video author (self-referenced builder; also notes T3.code is created by Theo and Conductor is associated with “vicat company” / another YouTuber, but the presenter is the primary source).
• Primary technical sources used:
  • Anthropic Claude Code Agent SDK (TypeScript SDK)
  • Claude Code / Cursor / Codeex agent SDK concepts
  • MongoDB (used + sponsored)
  • React + TypeScript + Zod (implementation tools)
  • Shared architecture comparison referencing T3.code vs Conductor