Video summary

Apple just shipped a 2nm chip... and nobody noticed the glue

Main summary

Key takeaways

Technology

Apple launches new Apple silicon (M6 and M5 Ultra) on a “2nm + AI compute” pitch

  • Apple positions the M6 as its first 2-nanometer chip and the M5 Ultra as a new desktop-class multi-die chip.
  • The pitch emphasizes AI compute and memory bandwidth, including a headline figure:
    • ~1.2 TB/s unified memory bandwidth on M6

M6 (consumer-targeted) — key technical changes

Process and CPU changes

  • 2nm process

    • Claimed benefits: “more transistors and smaller die,” enabling chips that are “faster and cooler.”

  • CPU: 12 cores total (+2 vs. M5)

    • Core breakdown is described as:
      • 2 “super cores”
      • 4 performance cores
      • 6 efficiency cores
    • Conceptual explanation
      • Single-thread speed benefits from the largest cores, due to:
        • reorder buffers
        • wide decode
        • large caches
        • higher power/frequency range (power-hungry voltage/frequency behavior)
      • Multi-thread throughput benefits from more efficient cores, because performance-per-watt drops at high frequencies.
      • Super cores target single-thread latency, while the remaining cores help throughput.

Neural Engine and GPU

  • Neural Engine

    • Dual 16-core
    • Claimed: up to 2× peak compute vs. the previous generation.

  • GPU

    • 12 GPU cores, each associated with neural accelerators
    • Claimed:
      • nearly 30% more peak GPU compute for AI vs. M5
      • much higher vs. M1
    • Introduces the phrase “prompt processing” as an acceleration-relevant area.

M6 performance analysis (what actually limits LLM speed)

Token generation bottleneck model

The video provides a practical bottleneck model for token generation:

Tokens/sec ≈ memory bandwidth / bytes of model weights read per token (i.e., generation is bandwidth-bound)

Example calculation

  • A 70B model quantized to 4-bit ≈ 35 GB of weights
  • With ~1.2 TB/s bandwidth:
    • theoretical ceiling: ~34 tokens/sec
    • realistic performance: typically “low 20s”

Key takeaway

  • GPU core count matters less for interactive chat/inference than memory bandwidth, because token generation waits on memory reads.
  • Prompt processing is more compute-bound (and more parallelizable), so accelerators can help there.
  • But the generation loop is the main limiter.

M6 “skeptical” purchasing constraints

Target audience vs real limits

  • Marketing target: “everyday users, students, developers, AI hobbyists, and enterprises.”

  • “AI hobbyist” realism check

    • ~32 GB unified memory is framed as a hard capacity wall for larger local models.
    • A 70B 4-bit model’s weights (~35 GB) won’t fit in that regime.
    • Even when it could fit, unified memory is shared—the OS/browser/editor also consume that same memory pool.

Bandwidth uplift vs capacity

  • Bandwidth uplift is presented as modest (around ~10%).
  • Capacity becomes the bigger constraint for local LLM use.

M5 Ultra — architecture framed as “gluing dies together” to bypass physical die-size limits

Why multi-die is needed

  • Lithography/reticle limits constrain maximum single-die size.
  • Larger dies have higher defect risk, reducing yield and increasing cost.

How M5 Ultra is built

  • Uses an Ultra Fusion-style approach (described similarly earlier for M1 Ultra).
  • Described as the first “quad-die” M-series architecture:
    • it fuses two dual-die M5 Max parts
    • resulting in 4 dies total (described as “layers of glued chips”)

Inter-die fabric and system behavior

  • ~4.4 TB/s interconnect, so the OS treats it like one logical processor.

Total resources and neural accelerators

  • Totals:
    • Up to 36 CPU cores
    • Up to 80 GPU cores
    • ~1.2 TB/s unified memory bandwidth (described as ~50% more than M3 Ultra)
  • The design adds neural accelerators to the GPU (unlike a prior comparison point where M3 Ultra didn’t).

Practical LLM fit guidance from the video

What developers should prioritize

  • Memory bandwidth + memory capacity
  • Treat CPU core count as “nearly irrelevant” for inference decisions.

Suggested model-size tiers

  • For 8B–14B models:
    • M6 Mini is claimed as strong performance per dollar for local AI.
  • For ~70B-class models:
    • A quad-die (M5 Ultra class) is framed as effectively required to keep the model in memory locally.

Data center comparison

  • The video compares Apple’s ~1.2 TB/s to HBM bandwidth on Nvidia H100, described as roughly about a third of the data-center level.
  • Conclusion:
    • Apple isn’t surpassing data center bandwidth.
    • But it may be “close enough” for some workloads to avoid cloud/GPU rental.

Overall conclusion

  • The “2nm” is portrayed as less important than the architectural workaround:
    • when you can’t make one huge reliable chip, compose multiple dies and connect them with very high bandwidth.
  • Apple is described as doing this “twice recursively” for a desktop form factor.

Main speakers / sources

  • Primary source: Shri Santhanam, Apple VP of Silicon Engineering (quoted/paraphrased for launch claims).
  • Secondary source: Apple’s newsroom/press release material for M6 and M5 Ultra specifications and marketing statements (including percent-ratio and architecture descriptions).

Original video