How I Watch ANY Video with AI in Seconds

Summary: “watch” video-understanding agent (key tech + product features)

• Core idea: The “watch” agent lets AI understand videos directly, which the speaker claims is faster and cheaper than methods that rely on hundreds of screenshots.
• The speaker says Google released a video-understanding agent that can answer “video comprehension questions” more accurately while using far fewer tokens.

Why this matters

Models don’t natively “watch” video content. Even strong models (the speaker mentions examples like Fable and Astra) can’t reliably infer what happens in a video by themselves, so the workflow adds a free “watch” function that supplies the model with the most relevant evidence from the video.

How it works (agent pipeline)

1. User input: Provide a YouTube link and a question/instruction to “watch.”
2. Backend: “watch” sends the link + question to Gemini.
3. Video review: Gemini analyzes the video, decides what parts to focus on, and returns an analysis with timestamps.
4. Multi-modal answering: The agent uses images + audio + transcript together to answer the request.
5. YouTube access claim: The speaker emphasizes no downloads and claims you “won’t get blocked” because YouTube is accessed via a Google-compatible API.

Integration: works across multiple AI tools

The speaker claims “watch” can be used via a view function so it works with “any AI tool,” including:

• ChatGPT
• Claude (via plugins/skills)

Demonstrated use cases

1) Find tiny details in long videos

• Example: A 3.5-hour Andrej Karpathy video.
• The user remembers the moment (token coloring + numbers) but not the tool name.
• They instruct “watch” to search the video for the demonstration and return:
  • tool name/website
  • timestamp
• The speaker verifies by opening the timestamp to confirm the correct screen.

2) Use retention/drop-off points to improve content (video analytics)

• Example: The speaker’s own “Book to Skill” video (~100k views) with retention data.
• Workflow:
  • Provide Claude the audience drop locations
  • Ask “watch” to analyze segments around those drops
• Claim: Claude can’t know viewer intent exactly, but can align drop points with content to identify likely patterns.
• Outcome claim: apply patterns across multiple videos to improve future scripts.

3) Auto-extract notes and save to an “LLM Wiki”

• Example: Again using the Karpathy video.
• “watch” extracts explanations + visual examples, then passes them to Claude to store as linked notes in an LLM Wiki (with an option to convert into documents).

Pricing / limits claims (Google free tier)

• The speaker claims the “watch skill” is free because:
  • It uses a Gemini model on Google’s free API tier.
  • The free tier includes up to ~8 hours of YouTube videos per day (as stated).
• If limits are reached, the agent:
  • waits for reset
  • doesn’t auto-charge
• Quota separation claim:
  • Claude’s own quota is used for processing the returned results
  • Gemini handles video analysis using Google’s quota

Installation / setup (high level)

ChatGPT Desktop / Claude Desktop (plugins/marketplace)

• Install the “watch” plugin/skill from the Watch repository (via marketplace/plugin menu).
• Claude desktop: Customize → Plugins → add repository/marketplace → install
• ChatGPT desktop: Plugins → Marketplace → install plugin
• Start a new chat and use /watch to begin setup.

Required credential: Gemini API key

• Create a key in Google AI Studio:
  • API keys → create API key
  • ensure the project is on the free tier
• Provide the key to the agent (or store it manually if preferred).

Update behavior

• If updating from a previous version, the speaker says preferences are retained.
• You can switch engines later (e.g., Gemini vs local transcription).

Engine options: Gemini vs Local transcription (Whisper X)

Default / recommended: Gemini engine

• “watch” performs video analysis and returns results, so you don’t need to review many frames.

Local option: standalone skill using Whisper X

• Uses Whisper X for local audio transcription, claimed to be free (especially when there are no subtitles).
• Local settings control:
  • how many frames the model sees
  • how many tokens are consumed
• Frame/token modes:
  • Efficient: fewer frames for a quick look
  • Balanced: frame selection based on scene changes
  • Token burner: maximum coverage (removes limits) but consumes much more Claude quota

When to prefer local

Local is preferred for tasks needing direct visual inspection, such as recreating motion graphics:

• Claude can inspect actual frames at timestamps
• You can compare frames to rendered output

Main speakers / sources

• Speaker: The video creator (not named in the subtitles).
• Referenced external sources/creators:
  • Google (Gemini + compatible API + video-understanding agent approach)
  • Andrej Karpathy (used in examples)
  • Claude (Anthropic model)
  • ChatGPT
  • Whisper X
  • TikTok/Instagram (supported sites claimed)
• Video referenced for an example request: Alex de Mozio (mentioned by name).