New DeepSeek Research - The Future Is Here!

Overview

DeepSeek published a large open-source research release (expanded from ~20 to ~80 pages) that the presenter calls a near-complete “recipe” for ChatGPT-like intelligence — freely available and reproducible (contrast drawn with OpenAI’s more secretive GPT-4 disclosures). The release describes practical training techniques that make high-performing language models cheaper, more private, and runnable locally (with sufficient hardware).

The presenter frames the release as a freely available, reproducible “recipe” for ChatGPT-like systems.

Five key technical takeaways

1. Group policy learning (GRPO — Group Relative Policy Optimization)

   • Instead of costly PPO with an expensive reward model/teacher, DeepSeek has the model generate many (e.g., 16) candidate answers per prompt and rank/compare them against each other (self-comparison).
   • This removes the need for a second large critic model and makes reinforcement training far cheaper and massively scalable.

2. Emergent “pause to think” / deliberation

   • Models learned to pause and deliberate (e.g., “Wait…”, “Let me re-calculate”) without being explicitly instructed.
   • The model discovered that spending more computation/time on reasoning improved reward and naturally increased its deliberation length.

3. Reinforcement learning / self-play over supervised examples

   • Pure RL/self-play (give rules, let the model explore) enabled models to evolve from poor performance to high competence on difficult math tasks, discovering new strategies without human examples.
   • Reported improvement: approximately a jump from ~15% success to nearly 80% on competition-level math tasks under this scheme.

4. Small amount of seeding (“flashlight”) helps steer learning

   • Starting with a few guiding examples (R1 vs R1 Zero) often dramatically improves performance on natural-language tasks (AlpacaEval) by preventing language-switching/gibberish behavior.
   • For abstract tasks like math, a few examples helped only marginally or not at all — math performance was largely language-agnostic as long as answers were correct.

5. Distillation (“learn from giants”)

   • The large R1 model produced ~800,000 step-by-step example “textbook” traces which were used to train much smaller models (distillation).
   • Result: small models (e.g., 7B parameters) trained from this distilled data significantly outperformed prior large models on competition math (claimed ~6× improvement vs a prior GPT-4o baseline on that dataset).
   • Implication: very capable models that run on laptops/phones soon, enabling private, local use without the billions in training costs.

Product / feature notes and practicalities

• DeepSeek model characteristics: smart, fast, reliable, private, and free to run if you have hardware; heavy GPU requirements but rentable (Lambda mentioned as the presenter’s rental provider).
• Distilled 7B model: small, high-performing, practical for local inference; signals democratization of strong LLMs.
• Datasets / benchmarks referenced: competition-level math tasks, AlpacaEval (natural language QA).
• Actionable strategies highlighted by the presenter: generate multiple solutions, pause and verify, and learn by doing.

Claims and implications emphasized

• Open-source reproducibility of ChatGPT-like training at scale.
• Cheaper training workflows that remove the need for costly reward models.
• Emergent reasoning/deliberation behaviors can be cultivated by RL signals.
• Distillation enables tiny models with large-model capabilities, accelerating availability of private/local LLMs.

Main speakers and sources

• Presenter: Two Minute Papers — Dr. Károly Zsolnai-Fehér (video host and commentator).
• Primary research/source: DeepSeek (their new paper/release).
• Comparisons / context: OpenAI (GPT-4 paper and GPT-4o baseline), AlpacaEval benchmark, and mention of Lambda (GPU rental service).