NCM: The LLM Killer?

Summary

The video examines two October research papers that aim to move beyond the standard language-model process of generating one token at a time: Next-Concept Modeling (NCM), which changes how text is generated, and Hybrid Reasoning Agents, which changes how an agent reasons.

Why move beyond token-by-token generation?

A conventional model generating a 12-token sentence typically takes 12 sequential generation steps, each depending on the previous one. Reasoning models can add many more steps because their internal reasoning is also represented as tokens. The video notes that these hidden reasoning tokens use context-window space and count as output under OpenAI’s guidance.

NCM: Generate a rough draft, then refine it

NCM represents text as a stack of coarse-to-fine layers. The top layer gives a rough outline; each later layer adds detail and corrects errors until the bottom layer represents individual tokens. Rather than writing from left to right, the model generates a complete layer in parallel during each pass.

• In a German translation example, the output improves over 11 passes, though an intermediate draft can sometimes be better than a later one.
• For news summaries, the authors report 99.8% accurate reconstruction of the layered representation.
• Training includes deliberately corrupted codes to help the model resist errors that accumulate during generation.
• Against similarly sized diffusion models, the authors report that NCM ran 31× faster on one news-summary task with a similar score, and 21× and 10× faster on two other text sets. One test used roughly five times fewer training tokens.
• In a Sudoku test using 25-million-parameter models, NCM solved 86% of difficult boards with 30 clues, versus 1% for a left-to-right model. The authors attribute this to NCM’s ability to revise the whole board across passes. However, the best diffusion model reached 81%.
• NCM performed worse than the diffusion model and a small left-to-right model on translation. Its largest tested model had 725 million parameters, so the results do not establish how it compares with much larger chatbots.

The authors suggest that future versions could choose the number of layers based on the complexity of the input.

Hybrid Reasoning Agents: Reason in hidden states, speak when needed

Hybrid agents alternate between ordinary text and silent reasoning steps. During a silent step, the model’s state is represented as numbers rather than readable words; a component called Hybrid Link feeds that state back into the model. The agent checks whether to continue silently or return to text—for example, to search the web or provide an answer.

In the example shown, the agent plans in text, reasons silently, searches for information about a company, and then answers “Mumbai.” The team automatically generated 54,000 training examples, and the model learned when to use silent steps without being explicitly rewarded for staying silent.

Reported results and caveats include:

• On GAIA, the authors report an average score of 52.4 for their 14-billion-parameter agent. The subtitles name a comparison model as “GPT 5.6 Luna,” with a score of 37.9, and report 6.8 for Coconut, an earlier silent-reasoning method. These are the figures as rendered in the subtitles.
• Compared with text agents trained in the same way, the authors report 35–55% fewer tokens, roughly 2× faster operation, and 2.7–3.7× lower cost. The cost is an estimate based on computing time, not an API bill.
• The approach did not lead on every benchmark: on the “Last Test of Humanity,” two leading models scored almost twice as high.
• In a math test, replacing silent steps with unrelated content reduced the 14-billion-parameter model’s score from 88.2 to 48.6. Replacing them with the closest dictionary words left the score nearly unchanged at 87.2. This suggests that the hidden states carry useful information, even when they are difficult to interpret as text.
• Because silent thoughts are accessible through the provider’s Hybrid Link rather than as ordinary text, they may be harder for outsiders to inspect. The video presents this as a transparency and control concern, alongside the potential intellectual-property rationale for keeping reasoning hidden. Peter Barnett criticized the approach as undermining a way to control AI and called it dangerous.

Availability and limitations

The video says NCM models are available under the MIT license, but the release does not include training code. Hybrid Agents released 8-billion- and 14-billion-parameter models and their training data. Using Hybrid Agents requires modified serving infrastructure to switch between text and silent states.

The video presents promising research results, not proof that either method is a general replacement for today’s large language models. It does not include a hands-on tutorial or product review; the evaluations and performance figures are reported from the papers’ authors.

Main speakers and sources

• Speaker: A single video narrator explains and compares the two approaches.
• Research sources: The NCM paper and the Hybrid Reasoning Agents paper, along with their authors.
• Other sources mentioned: Google Research on decoding steps; OpenAI’s reasoning guidance; Meta’s Coconut work; and Peter Barnett’s public response.