The Hidden Failures of AI

Overview

The video is an interview focused on how modern AI systems fail “quietly,” and what product developers can do to detect and correct those failures—rather than relying on users to complain.

Key points and main arguments

• Most AI failures are “invisible.” Chris Potts says that after analyzing about one million ChatGPT conversations, his team found ~78% of AI failures leave no trace—meaning the user doesn’t explicitly signal that something went wrong, even though the response may be incorrect, incomplete, or mismatched to intent.

• Why invisibility happens (and what companies do today). Potts explains that many systems are not robust at interpreting nuanced user dissatisfaction. He gives an example: a profane regular expression reportedly embedded in leaked code meant to detect frustration signals (high-precision but low-recall). This illustrates that companies often detect only a narrow subset of failure signals.

• Monitoring should look for failure patterns beyond user complaints. The core claim is that developers should be able to detect issues by auditing the interaction itself and by using model-based self-checking/verification. “Invisible failure” includes cases where the AI’s answer is not actually responsive to the user’s question—the system produces a plausible output that still doesn’t resolve intent.

• Self-checking and multi-model critique can improve reliability—especially when real verification is possible. The discussion compares prompting the model to grade its own answers, and approaches where one model checks another. Potts agrees this is generally productive, but emphasizes that it ultimately needs to connect to stronger forms of verification. He contrasts domains:

  • In software, verification (running the code) is direct and scalable.
  • In harder domains like legal/UX, verification is less straightforward, so accuracy/hallucination problems remain harder to address.

• The company’s product thesis: an “audit/monitoring layer” for AI interactions. Potts describes his startup BigSpin.ai as building an app/agent that helps organizations observe and improve AI product behavior through:

  • detecting low-level “signals” of failure,
  • modeling user expertise and task/domain context,
  • and suggesting fixes to connect product and engineering teams. A key differentiator is that BigSpin uses many automated model-based annotators (treated as specialized classifiers/agents) to detect “invisible failures” and context signals.

• User behavior matters: “delegative mode” vs “augmentative mode.” Potts argues many users operate in a passive delegation style (accept answers at face value) rather than an expert-like augmentation style (critique, double-check, ask for contradictions, iterate). Users complain about hallucinations, but often don’t take the steps that would correct them.

• Failure archetypes and interaction loops can worsen problems. The interview references Potts’s earlier research framing failures into archetypes (including):

  • confidence trap (seeming confident but wrong),
  • drift (the system follows the goal imperfectly),
  • walk-away / non-resolution (response doesn’t actually answer what was asked),
  • death spiral (repeated attempts with no real progress, followed by quitting). He also expresses concern about agent loops/autonomy: if systems keep iterating without opportunities for correction, they can become expensive and erratic.

• Not “stochastic parrot” pessimism; still cautious optimism. While the conversation acknowledges failure modes, Potts rejects the idea that LLMs are simply dumb/unproductive. He argues that the models are doing sophisticated learning and that progress will continue—though product reliability requires better detection/verification.

• Broader linguistics/understanding debates: what “understanding” might mean. Potts discusses the philosophical and scientific ambiguity around whether models “understand,” suggesting the topic should be treated cautiously and with an open but grounded stance. He frames “understanding” as a potentially meaningful mapping from language into conceptual structures, with current models likely behind human-level mapping in sophistication but not blocked in principle from improving.

• A linguistic tool for AI research: comparing languages and internal representations. Potts answers questions about whether models trained on different languages (e.g., English vs Chinese) develop qualitatively different internal structures—citing differences due to tokenization and the conceptual mapping induced by training data. He notes that cross-lingual influence can lead to observable biases (e.g., how color terms might be used when English is influential).

News/commentary themes

• A practical “AI safety/product quality” theme: invisibility of failure is a major operational risk.
• A business theme: value isn’t only in the base model; it’s in interaction-level auditing, labeling, and context-aware monitoring.
• A research theme: interpretability/verification and specialized evaluation pipelines are central to making AI dependable.

Presenters or contributors (all mentioned)

• Interviewer/host: Mike (appears repeatedly as “Mike”)
• Guest: Chris Potts (Stanford linguistics professor; founder of BigSpin.ai)
• Other individuals referenced in discussion:
  • Omar Khattab
  • Matei Zaharia
  • Stephen Colbert (in a name/pronunciation aside)
  • Rich Sutton (referenced via “bitter lesson”)
  • Yann LeCun
  • Fei-Fei Li
  • Geoffrey Hinton
  • Leo (referenced as “Leo” in the segment)
  • Ruben Schatner (book/editing mention)
  • Nita (the producer)
  • Aux Demost (referenced)
  • Jeff (referenced in the “LeCun camp / Fei-Fei Li camp” context)
  • “Omar happily accepts Colbert” (Omar Khattab again)
  • “Nostalgia Nerd” (author of The Void, recommended)