Jonathan Blow on why LLMs cannot program

Overview

Jonathan Blow argues that today’s LLMs cannot genuinely “program” in the way real software development requires. While LLMs can be useful accelerators—especially for repetitive or boilerplate web coding—they lack the reasoning and understanding needed to build and maintain complex, correct software systems.

Key Points

• LLMs excel at pattern-matching for trivial/repetitive tasks: Much of web development involves rewriting similar simple operations repeatedly, and LLMs are well-suited for that kind of work.
• Programming requires invariant-preserving changes: Real software maintenance depends on understanding and preserving invariants (the underlying rules that must remain true for the system to keep functioning). Blow claims LLMs “wing it” by rewriting code without truly understanding those invariants.
• They can “pretend” better than they actually reason: Blow emphasizes that LLMs can generate outputs that resemble programming, but this is not the same as the deep reasoning and careful change management required for reliable software.
• Progress is real—but not magic, and not unconstrained: He acknowledges LLMs are an advancement, but rejects hype that implies they will automatically overcome limitations just because they keep improving.
• Other AI approaches may help more than LLM hype: Blow suggests progress has also come from non-LLM AI systems, and that effective programming assistance may require “scaffolding” (additional structure/tooling), not just better text prediction.
• “Scaffolding” won’t eliminate fundamental issues: He uses an analogy to argue that adding wrappers can’t fundamentally transform the core capability in the way some people assume—i.e., you don’t turn a “donkey into a horse” just by claiming improvements will eventually make it equivalent.
• Critique of “eventually it will solve everything” arguments: He criticizes the reasoning that if LLMs keep getting better, they must eventually do what people want, calling it unsound and driven by treating the tech like magic.

Presenters / Contributors

• Jonathan Blow