Why are AI companies hitting the brakes? | DW News

Overview

AI leaders and researchers argue that major safeguards for advanced AI are not keeping up with rapidly improving capabilities. They contend that the industry may need to slow or reshape development to reduce existential and catastrophic risks.

Key arguments and reported developments

Calls to “slow down” AI (and concerns safeguards are failing)

• Anthropic CEO Dario Amodei warns that existing safety measures may not be sufficient to control the pace or potential of advanced systems.
• He predicts that within a year, groups of AI agents could cause catastrophic damage online.

OpenAI delays and unusual cross-industry alignment

• Sam Altman (OpenAI) says OpenAI will push back its IPO to next year, citing AI risk concerns.
• Altman and Elon Musk have supported Amodei’s “slow down” call, suggesting broader internal alarm across major competitors.

Control AI’s position: ban “superintelligence,” not just tools

• Conor Leahy (Control AI) argues that proposed policies should target fully autonomous “superintelligence”—systems that could outperform humans across economics, science, engineering, politics, and warfare.
• He claims these systems would be not reliably controllable or understandable.
• He supports a ban on developing such systems domestically and internationally, warning that risk persists if any country or company builds them.

International “trust but verify” enforcement model

• Leahy argues a US-only ban is insufficient, since other countries (e.g., China) could race ahead.
• He supports international regulation with:
  • Verification mechanisms
  • Credible deterrence
  • Escalation steps such as political/economic sanctions up to self-defense

Safety vs. “marketing hype” critique

• Leahy criticizes industry messaging as implying “pacing” rather than stopping.
• He argues that, given the stakes, companies should halt development if there is meaningful risk—citing internal assessments that suggest a nontrivial chance of catastrophic outcomes.

Why warnings are emerging now (Carnegie fellow’s analysis)

Sam Winter Levie (Carnegie Endowment) says two key factors explain the current surge in warnings:

1. Capability leaps: models increasingly solve difficult problems and can threaten parts of the internet and modern technology.
2. Uncertainty about control: researchers are not confident builders fully understand or can manage what these systems will do.

Geopolitics and regulation challenges

• Levee argues companies cannot be trusted to regulate themselves, requiring government action.
• He notes governments still need to catch up on technical expertise and political consensus.
• He also flags potential US conflict of interest, as leaders weigh AI risk against fear of losing to China.

“Pacing” the frontier vs. a full pause

• If slowdown occurs, Levie expects labs would likely:
  • Reduce the release rate of new models
  • Increase testing, alignment checks, and real-world behavior evaluation
• Public progress may still look rapid, but internal safety efforts would intensify.

Why insiders are speaking now despite long-known risks

• Levie argues the risks may have been discussed earlier, but exponential progress makes the danger clearer once capabilities cross practical thresholds.
• He also suggests some firms previously prioritized fear of falling behind rivals; that concern may have shifted toward fear that systems could harm people in unforeseen ways.

Overall takeaway

The video highlights a rare convergence among prominent AI figures: despite competition, leaders increasingly believe advanced AI development is outpacing control measures. The central debate is whether governments should impose binding international rules, especially since unilateral slowdowns could be undercut by rival states.

Presenters or contributors

• Dario Amodei (Anthropic, CEO)
• Sam Altman (OpenAI)
• Elon Musk (xAI)
• Conor Leahy (Control AI, US Executive Director)
• Sam Winter Levie (Carnegie Endowment for International Peace, Senior Fellow)