Video summary

Google has secretly been watermarking your text for years.

Main summary

Key takeaways

Technology

Key tech/product concepts and what the video claims

  • AI text watermarking is becoming required for EU-facing AI labs. The video claims that Anthropic’s watermark for Claude is driven not just by product preference, but by new EU law requiring labeling of AI-generated content—including text and even code.

  • Watermarking is “invisible” but detectable later. The watermark is embedded into the text itself (not appended as “generated by Claude”). It’s said to work using patterns in token selection.

  • Anthropic’s approach is based on a paper credited to Google DeepMind / Gemini. The video claims Anthropic publicly states its method is based on an earlier Google/DeepMind watermarking technique, described as similar in general strategy to Gemini invisible watermarking.

  • Google has used a similar system already. The video claims Google has been watermarking for about 2 years, referring to it as SynthID, analogous to how Google watermarks images.

How the watermark works (technical intuition + details)

Token generation basics in LLMs

  • LLMs typically generate tokens using probabilistic sampling (e.g., top-p/top-k filtering, then sampling from the remaining options).

Making sampling “controlled”

  • Watermarking changes randomness so it becomes pseudo-random but structured:

    • Instead of purely random sampling, the generator uses a pseudo-random mechanism with a secret key.
    • The watermark authority (the party with the secret) can later recompute expected properties of the token stream.

  • Intuition offered in the video: it’s like using a deterministic-but-random-looking sequence (e.g., digits of π) versus rolling a fully random die—predictable to the secret holder while still appearing random to others.

Tournament-based selection mechanism (from the referenced paper)

  • The method is described as “tournament-like”:
    • After sampling constraints are applied, there is a competition among candidate next tokens.
    • A Bernoulli-style function yields a 0/1 “G value” for each token based on:
      • a secret key
      • previous tokens
      • a layer index (the paper is described as using multiple layers; the speaker claims 30 layers)
    • The “winners” are tokens that tend to have higher G values across layers, producing text where many tokens show statistically detectable properties.

Detection by re-evaluating G-values

  • A party with the secret key can examine text and estimate whether a large fraction of tokens match the expected high-G pattern.

Detection strength depends on length

  • The video argues detection is weak for very short outputs, since a small number of tokens can accidentally produce the “right” statistical pattern.
  • It becomes more reliable for multi-paragraph responses.

Implications for quality and detectability (including code)

Claimed quality impact is minimal / none

  • The video claims watermarking does not meaningfully reduce quality, citing the idea that sampling is already stochastic and watermarking only changes which randomness is used.
  • It references a Google study using about 20 million Gemini conversations, reporting no discernible quality impact via thumbs up/down feedback.

Code detection is expected to be harder

  • For code, token distributions can be extremely skewed. For example, a closing parenthesis might be ~99.9% likely.
  • In such cases, the tournament may produce ties (because candidate tokens are overwhelmingly similar), potentially reducing the statistical signal available for watermark detection.
  • The speaker concludes that AI-generated code may be harder to detect than AI-generated emails using this approach.

Review / guide / tutorial elements

  • The video is not really a guide to watermarking, but it includes a recommendation on how to choose models.
  • The speaker recommends HubSpot’s free “AI model cheat sheet bundle”, including:
    • an LLM selection cheat sheet (what models are good at, limitations, and when to use them)
    • a task-to-model decision matrix (task mappings to models, including cost considerations and exploration of open-weight options)

Main speakers / sources

Speaker

  • A single primary narrator/creator discussing Anthropic, Google, and the referenced watermark paper.
  • The video later references the speaker’s experience, including “spent 6 years at Google” and “6 years.”

Primary sources referenced

  • Anthropic: public statements about watermarking Claude
  • Google DeepMind / Gemini: the earlier watermarking paper and the SynthID system
  • EU law: legal requirement for AI labeling
  • HubSpot: AI model cheat sheet resources

Original video