Harvard Just Discovered AI Isn’t Actually Working

Summary

The video argues that AI is making some tasks—especially writing software—faster, but that this does not automatically make entire organizations more productive or eliminate the need for workers. Its central idea is “time displacement”: when AI speeds up one stage of work, the time saved can be absorbed by review, coordination, approvals, testing, and other bottlenecks.

Studies and productivity findings

The narrator contrasts two studies of AI-assisted software development:

• An initial METR randomized trial, involving 16 developers and 246 tasks, reportedly found that developers using AI took 19% longer.
• A later attempt to repeat the study struggled to recruit developers willing to work without AI. The narrator says this created selection bias: developers who felt they benefited most from AI were less likely to participate in the non-AI comparison.

The video also describes a Harvard analysis of data from 718 companies, including Jira tasks, GitHub activity, merge requests, deployments, and calendars. It reportedly covered 300 million data points over five years.

Companies using AI coding tools and agents reportedly:

• Wrote 30% more lines of code
• Committed 20% more code
• Opened 23% more merge requests

However, the study found little evidence that firms completed more Jira tasks or projects, or substantially increased the amount of finished work.

The narrator says engineers spend about 42% of their time coding, 18.6% reviewing code, and 39.5% in meetings, documentation, and other coordination. AI can accelerate coding, but work may then shift to downstream tasks:

• Merge requests reportedly took 49% longer to approve.
• Time spent fixing code during review doubled, while reviewers left 35% more comments.
• Testing, legal approval, product coordination, and QA remain human-dependent bottlenecks.

Workplace risks and examples

The narrator describes companies spending heavily on AI tools and tokens, while also citing layoffs attributed to AI. The video questions whether these cuts reflect proven productivity gains or whether executives are using AI as a justification to reduce headcount.

It gives examples of companies cutting jobs or offering severance to employees despite strong financial results. The narrator argues that AI-driven efficiency claims can create job insecurity and a more stressful workplace.

An Amazon outage is presented as an example of the risks of AI-generated code. According to the video, the incident led the company to add safeguards, including a security reboot, senior-engineer approval for junior employees’ AI-generated changes, and required documentation. The narrator characterizes these measures as added friction intended to prevent costly errors.

The video also notes that AI’s effects extend beyond software:

• A JAMA study of AI medical documentation tools reportedly found reduced overall record-keeping time, but doctors using them spent a larger share of appointment time completing charts, partly because AI notes needed review and editing.
• The narrator cites legal and accounting cases involving AI-generated false material, including hallucinated legal citations and fabricated links or quotations in a report.

Overall conclusion

AI is portrayed as a powerful tool for accelerating specific tasks, not as a complete replacement for human work. Faster code generation does not necessarily mean faster delivery, higher overall output, or fewer workers: the rest of a production process still has to absorb, review, test, and approve the work.

The narrator’s concern is that workers may lose jobs because executives invoke AI as the reason for cuts, even when the technology has not removed the organizational bottlenecks that limit productivity.

The video is an analysis and commentary rather than a product guide or tutorial; it does not provide step-by-step instructions for using AI tools.

Main speakers and sources

• Dylan John, the narrator, who identifies himself as an AI engineer and discusses his workplace experience
• Research attributed to Harvard, METR, and JAMA
• Company statements and reporting about Amazon, Microsoft, Meta, Netflix, American Express, and AI-tool adoption
• Anecdotes from engineers and other workers quoted or paraphrased in the video