Video summary

Pi Setup After 6 Months of Use

Main summary

Key takeaways

Technology

Tech/Product/Workflow Summary (Pi setup after ~6 months)

1) Updated Pi Agent environment (major shift since prior config)

  • The creator notes that the setup has changed substantially since an earlier Pi configuration video (about ~4 months earlier).
  • Key links are provided in the video description (not detailed in the subtitles), including a “training system” referenced from the previous video.

2) Extensions & tools

Bash Guard

  • A small change: adds an “off” switch for easier session control (the creator typically disables it as the first step).

User Query Tool (unchanged core, improved coordination)

  • Core functionality remains: the AI can ask the user questions.
  • Improved coordination prevents popup-based extensions from overlapping, enhancing multi-extension UI behavior.

Custom Header + Web Tools

  • Both are effectively unchanged from the prior setup.

3) Interactive asynchronous subagents (big new feature)

This feature replaces the creator’s older synchronous subagent approach.

  • Key capability: Subagents run in the background and remain interactive via terminal multiplexers.
  • Requirement: tmux (subagents are launched inside tmux panels).
  • Fast tool-call behavior: The tool call returns immediately while the subagent session continues in the background.
  • Agent-to-agent interactivity:
    • Subagents can communicate with their parent/orchestrator agent.
    • Question/clarification routing: subagents have a question tool; when a subagent asks a question, it “wakes up” the orchestrator/parent agent to deliver the answer back instantly.
    • Demonstrated a multi-level delegation chain (e.g., scout → executor → master/orchestrator), including standby mode while waiting for answers.
  • Completion & session recovery improvements:
    • Agents use names to route messages/tool calls.
    • Recovery/management uses the same tool mechanism.
    • Built-in subagent “completion” handling reduces reliability issues seen in the older approach (where a separate completion tool was sometimes missed or called too early).
    • Supports multiplexer restoration (e.g., resuming the first demo scout session).
  • Observability: Shows real running sessions directly in front of the editor/chat area (“maximum observability”).
  • Origin: A fork of Daniel Grieser’s interactive subagents, with the creator simplifying UI/layout and combining interrupt/resume notification into a single tool.

Demonstrations mentioned:

  • A “scout” demo showing tool-call immediacy + interactive background session.
  • A toy task: “solve the Riemann hypothesis” showing parallel activity (~7 subagents).
  • Auto-balancing of panels (layout management improvement).

4) Browser extension (Playwright-based headless browser)

  • Purpose: enables Pi to perform browser tasks using a headless Playwright workflow.
  • Adds eight tools (agent-visible tools affecting context size).
  • Disabled by default; must be enabled explicitly with “browser on”.

Demonstration workflow:

  • Visits a site (e.g., example.com), runs JavaScript, and takes screenshots.
  • Limitation noted: the model can’t view screenshots/images (“GLM forgot that he has no eyes”).
  • A broken web application debugging scenario:
    • With browser tools enabled: screenshot + console inspection + error detection.
    • Then the agent continues interacting (click testing) and validating the fix.
  • Emphasis: the agent doesn’t need to rewrite the entire site—just locate and address the issue.

5) Dictate extension (speech-to-text)

  • Used throughout the video.
  • Transcribes speech into text using a “minimal Whisper flow” (details referenced as covered in another video).

6) Observational memory extension (custom design inspired by “Mostra”)

  • The creator’s own implementation (they mention an existing strong direct implementation elsewhere, but they built their own).
  • Core idea: Convert message history into deterministic memory “observations” extracted by observer agents.

How it works (conceptually)

  • Message history → fragmented → observer agents extract atomic observations.
  • Observations become the unit of memory.

Deterministic compression

  • Compression is deterministic, avoiding the “resume summary effect” common in LLM-based compression (where summaries degrade or get reinterpreted over cycles).

Long-running session strategy

  • Once the observation pool is full, the oldest observations are consolidated into Markdown files by topic.
  • A memory index is provided to the agent at the start of compression blocks.
  • Defined memory layers:
    • Observations = short-term memory
    • Topic-based Markdown files = long-term memory
    • A compressed “tail” of messages = working memory

UI/telemetry shown in demo

  • “/OM” enables the system and shows “meters/sensors.”
  • Metrics include:
    • When the next observer appears (every ~10,000 tokens)
    • Progress toward consolidation/compression
  • “/OM status” outputs a full timeline of session blocks (~10,000 tokens each), with color coding for:
    • consolidated observations
    • observed but not yet consolidated
    • not yet observed
  • Demo behavior:
    • Compression occurs instantly in-session (no need to pause).
    • Shows:
      • observer created a set number of observations (e.g., 14)
      • later, the consolidator combines more observations (e.g., 85) into memory documents
      • memory pool advancement
      • a “travel block” (minimal token-limited navigation structure) edited by the consolidator agent

7) Prompt fragments (new prompt composition mechanism)

This addresses a problem: where behavior instructions should live.

  • System prompts / agents.md:
    • Always present but hard to discover/edit.
  • Skills:
    • Situational but bulky
    • Often enabled by default; the creator prefers more explicit control

Prompt fragments solution

  • Very small standalone behavioral instruction snippets that can be combined.
  • Activated via Alt+S snippet menu:
    • select snippets and preview them as they’re added to the beginning/end of a prompt.
  • Stored/edited as Markdown files in a folder.
  • Snippets can be ordered and are intended as a middle ground for instruction strength.
  • Rationale: instructions may carry more weight when placed directly in the message rather than only in the system prompt/skills.

Examples used:

  • “ask questions”
  • “verify, don’t assume”
  • “orchestrator mode”
  • “questions”
  • “research delegation”

8) Skills (reusable capabilities)

  • Includes items from prior videos (PDF reader, YouTube transcript).
  • Contains a web debugging skill tied to the browser extension.
  • Mentions a “session analysis skill”:
    • Python scripts for analyzing/searching Pi sessions
    • Used to find repeated instruction patterns and convert them into the selected prompt fragments.

9) Training system (released/linked)

  • The creator stresses that demand for a training system is high.
  • Goal: release it alongside the rest of the Pi configuration.
  • Options:
    • Recreate the exact setup from the video (requires additional extensions)
    • Try it quickly without subagents (simpler alternative)
  • Encouragement: viewers should copy ideas and adapt to their workflows; the creator plans continued development based on feedback/comments.

Main speakers/sources (as stated in subtitles)

  • Primary speaker: the video’s creator (the person giving the walkthrough of their Pi setup).
  • Referenced prior author for subagents fork: Daniel Grieser (interactive subagents).
  • Mentioned technology/library: Playwright (via the browser extension).
  • Mentioned model/tech: Whisper (via the “Dictate” transcription flow).

Original video