GitHub Senior Engineer: Software Engineering Is Broken! Here's What's Next

Overview

The episode argues that software engineering is being pushed in an unsafe, unhealthy direction by over-relying on AI across the entire development lifecycle. This includes:

• Automated PR generation
• Automated AI “reviews”
• Implied/auto-approval mechanisms

The concern is that before high-stakes systems can reliably handle correctness and accountability, teams are outsourcing too much judgment to AI.

Core worry: stacking imperfect systems (agentic code + LLM review) increases societal risk while making responsibility unclear.

“Octopus” AI Engineer Mindset

The guest defines an AI engineer as someone who combines skills from multiple disciplines:

• Software engineering: build real applications
• DevOps: deploy and operate systems
• ML/LLM integration: connect models to products and workflows

Rather than only training models from scratch, the practical goal is to apply AI end-to-end to solve real problems.

What’s Broken With Current AI Use

The main critique is that teams are stacking imperfect components:

• agentic code generation
• LLM-based “reviews”

This may appear workable for low-risk apps, but the episode highlights societal risk for enterprise-critical domains, such as:

• hospital systems
• airline systems

Accountability Gap

A key theme is accountability. If an AI-approved change causes harm, it’s often unclear:

• who is responsible
• who signed off
• how liability is determined

Why the Guest Left GitHub for AI Safety

The guest frames their resignation as a response to seeing more startups and workflows replace real review with LLM-based PR review.

They wanted to work on new methods that could make the “agent reviewing agent / auto-approval” pipeline safer.

How AI Should Be Used (Hybrid + Context-Dependent)

The guest supports AI coding, but insists on:

• human judgment
• scaled accountability

They reject extremes:

• Don’t ban AI entirely
• Don’t allow agents to ship code with no review

Risk-Based Constraints

In higher-consequence environments, autonomous coding should be more constrained than in:

• freelancing
• greenfield/product experiments

Lower accountability settings shouldn’t automatically justify higher autonomy.

“Detox” From AI (Interviews + Skill Retention)

After using AI daily, the guest had to stop temporarily for a safety-lab interview process where agents were banned.

They describe this as a 1.5–2 week “AI detox.”

During the detox they re-practice fundamentals (e.g., coding/LeetCode-style refresh) because manual fluency can degrade.

Interview Integrity and AI Cheating

The guest describes hiring pipelines encountering candidates with deepfake-like characteristics—realistic profiles paired with suspicious camera feeds/behavior.

They believe cheating can be mitigated when:

• senior engineers ask critical questions
• hiring includes human verification

They also warn that “AI-only hiring” is risky: systems should assess candidates’ actual understanding, not just their ability to produce outputs.

What Companies Should Do About Cheating

Suggested approaches include:

• At least one in-person round (or an equivalent way to evaluate real behavior and culture fit)
• Internships with real goals and evaluation (harder to fake over months)
• AI-usage rounds designed to test how candidates work with AI tools, rather than relying only on “no-AI” rules

A balanced process can measure both:

• fundamental knowledge
• effective AI tool use

Pivoting Into AI Engineering (Market-Facing Guidance)

1) Soft Skills Matter

Particularly the ability to explain technical concepts and AI research to non-technical stakeholders, across:

• horizontal scope (explaining across different personas)
• vertical depth (owning key AI concepts)

2) Projects Beat “Slop”

Build projects tied to personal subject-matter expertise, with evidence that it works, such as:

• evaluations
• measurable impact
• structured proof beyond “deployed to Vercel”

3) Proof and Evaluation Are Crucial

Examples include:

• test suites
• metrics
• structured validation of outcomes

Skills Roadmap

Lasting Foundations

• linear algebra
• language-model fundamentals (tokens/context)
• core patterns like RAG, which the guest argues is not “dead” and remains widely used

Full-Stack / DevOps Capability

Beyond calling APIs, you need the ability to deploy and operate real systems.

Why Linear Algebra Still Matters

Math doesn’t need olympiad-level ability, but basic linear algebra is important because AI-generated math/code can be wrong—and must be interpretable.

Learning With AI vs Humans

The guest argues AI tools (e.g., ChatGPT/Claude) can help, but they can’t guarantee:

• up-to-date, experience-based strategy

Human coaching is framed as especially valuable for:

• current job-market tactics
• non-generic guidance

Presenters / Contributors

• Otto Brazil — interviewer/host
• Zeno — guest (GitHub Senior Engineer; later at an AI safety lab)