Dear YouTube!

Overview

The speaker argues that YouTube’s newly announced “video A/B testing” feature is an unusually risky—and potentially harmful—idea. They place it within a broader pattern: YouTube makes changes aimed at competing with other platforms rather than doubling down on what makes YouTube distinct.

---

Main Points of the Critique

1) It breaks the shared “same video” community experience

YouTube has an unspoken social norm that viewers are watching essentially the same upload together. Allowing multiple simultaneous versions (e.g., 2–3 cuts) undermines that shared cultural experience.

2) Viewer and commenter confusion

• Viewers likely won’t know they’re watching a test version.
• Timestamped comments could become misleading or “broken,” since different viewers may be seeing different cuts.
• Comment authors won’t necessarily know which version others are watching, increasing the chance of context drift.
• A viewer could watch a version that “loses,” and later see a different version if the test changes or concludes.

3) Unclear boundaries and validity of results

• The speaker questions how different the tested videos are allowed to be (e.g., small intro changes vs. major structural changes).
• They suspect small differences—similar to past thumbnail testing outcomes—often don’t produce meaningful statistical separation. Forcing A/B testing on minor edits may therefore not be genuinely informative.
• They also worry about how YouTube will handle cases where seemingly minor segment changes alter the overall message/structure (including timeline order).

---

What YouTube Allegedly Plans (Based on an Engineer Call)

• YouTube may have had extremely limited “video replacement” capability behind the scenes, but A/B testing at scale is different.
• For rollout to everyone (early next year), YouTube plans to use a semantic similarity analysis model to compare uploaded video files and only allow tests where versions are sufficiently similar.
• A/B testing will be limited to the initial launch window of the video; creators won’t be able to revisit older uploads and repeatedly re-test different cuts.
• However, many practical questions remain unanswered, especially:
  • How the similarity model behaves in edge cases
  • Whether it reliably blocks materially different videos

---

Broader Trend Argument (Why It Fits a Pattern)

The speaker contextualizes the announcement alongside other changes they believe are driven by competitors:

• Removing the public dislike counter (framed as protecting creators, but aligned with other platforms).
• Changing how views are counted—now counting when the first frame loads—which the speaker argues makes view counts less comparable to earlier, stricter methods.

---

Advice and “Hot Take” to Creators

• Most creators should not use the feature.
• The speaker claims A/B testing mainly optimizes retention metrics, not necessarily creativity or overall video quality.
• They argue that heavy optimization could reduce niche, characterful, or creative choices that don’t maximize efficiency but make YouTube compelling.
• They recommend focusing on making and finishing one strong upload, learning from results over time rather than iterating many variants to squeeze incremental retention gains.

---

Presenters / Contributors

• Primary presenter (speaker): Unspecified (referred to as “I” / “the speaker” in the subtitles).