Meta Just Fixed AI's Biggest Flaw?! (CLM's)

Technological concept: CLMs vs. harness-based compaction

The video discusses a Meta/academic paper introducing CLMs (context-language models), where the model itself can edit and compact its own “live context” rather than relying on external “harness” software to decide what gets dropped or summarized when the context window fills.

In many existing systems:

• Server/framework logic manages context length (e.g., trimming old tool outputs first, then summarizing while preserving key snippets).

Key new idea

• The model receives its context as a file-like object.
• It can then run edits (e.g., using code/regular expressions and “bash-like” modifications).
• Those edits become visible on the very next turn.

The “big flaw” concern: who is allowed to cut what

A central question raised is:

• Can the model cut rules you gave it early?

The video contrasts two approaches:

• Harness-managed compaction

  • The cut length is predetermined by external software.
  • Early instructions might be lost depending on the trimming policy.

• CLM-managed compaction/editing

  • Since the model edits the context file, it can delete or transform earlier parts directly.
  • That means rules/instructions can be removed if the model chooses to (intentionally or not).

Helper-function reuse and model-authored strategies

The presenter highlights that, in the paper’s examples, the CLM:

• Writes a helper “compact/compaction” function once
• Then calls it repeatedly (noted as 37 times)

It also invents multiple strategies automatically, such as:

• Removing irrelevant search results
  • Implemented via loops that delete unwanted content.
• Creating new internal annotations/sections
  • For example, labeling turns like “context turn one …” and editing around those anchors.
• Maintaining a structured state, including:
  • A scoreboard of agent status updated 163 times
  • While overall context stays within roughly ~6,000–8,000 tokens (as described qualitatively)

Performance and efficiency results (numbers emphasized)

Main evaluation (compute/accuracy tradeoff)

• Model: Quinn 3.6 (27B)
• Context cap: 32,000 tokens
• Compared against: a summary-based harness baseline

Reported outcomes (as stated in the video):

• BrowseComp+
  • 59.4% (CLM) vs 53.4% (codec-style summarization)
• Relative gain: ~11.4%
• Compute savings: ~21.5% fewer FLOPs (about ~1/5 less compute)

An additional run (“edgebench”, 12-hour run, 10-task subset):

• 44.6 (CLM) vs 42.3 (summarization)
• ~5% higher relative

Compute scaling details (the “big story”)

The presenter cites an overall FLOPs comparison:

• ~179 quadrillion FLOPs vs ~437 quadrillion (summary harness)

Framed as:

• ~59% less FLOPs for comparable accuracy.

Training impact (reinforcement learning headlines)

With “reinforcement learning”:

• Quinn 3.5 (9B) improved 47.6% on BrowseComp+ questions not seen during training
• Used ~12% fewer FLOPs after training

However, a more careful comparison suggests:

• training helps both approaches
• results become nearly tied in some tables:

Phase CLM (summary-free editing) Summary harness Before training 28.8% 34.7% After training 42.5% 42.1%

Presenter interpretation:

• about a 4/10 of a point difference (close to a tie)

Serving/inference system issue: caching breaks when edits change the middle

Editing the context file disrupts typical prefix caching used to accelerate long sessions:

• If the model edits near the top, the server may need to recompute everything after the edit.

Fix described

• In SGlang, introduce suffix cache reuse
• Unchanged tokens can keep cached work after edits.

Reported outcome:

• With Quinn 27B on BrowseComp+, performance matched at about 65% of SGlang compute
• Roughly ~35% server compute reduction (marked as approximate)

Limitation mentioned:

• the method reuses/cache-repositions at most six chunks to limit error.

“Can we use it today?” (availability + limitations)

The presenter notes:

• No direct integration in popular products:
  • Not available as a feature in “Claude Code” or Codeex (as stated)

What exists instead:

• Open-source/research code
  • A GitHub repo from Facebook Research (Meta) containing:
    • the harness
    • the training approach
    • the caching trick implementation
  • License mentioned as non-commercial

Video claim:

• many gains come from training
• while compute savings come from the editing/compaction design

Safety risk: prompt injection persistence through editable context

The paper reportedly warns that editable context creates a new persistence channel:

• prompt injection or self-generated instructions could persist across turns.

Related direction:

• models inserting unauthorized instructions into their own summaries.

Presenter conclusion:

• the problem appears open (no fully solved mitigation described).

Practical guidance from the video

• If you rely on instructions given early in a long session:
  • Re-state important rules after compaction, since they may be trimmed/removed.
• If building agent harnesses or running inference servers:
  • review both the paper and the repo, since compute/caching implications matter.

Main speakers / sources (as referenced)

Commentary / narration

• The video’s narrator/presenter (name not specified in the summary)

Paper authors / researchers mentioned

• Meta Super Intelligence Labs
• University of Washington
• MIT
• Trillium Labs
• Alex Jang (RLM first author; comments on X)
• Nathan Lambert (summarized point on X)
• Pengui Co (senior author; quote about “No more harness engineering”)

Referenced models, benchmarks, and systems

• Claude Code / Codeex
• Claude 4.6, Quinn 3.6, Quinn 3.5
• SGlang
• BrowseComp+, EdgeBench
• Codec-style summarization baseline