"Я расскажу это впервые". Почему ушел из OpenAI, зачем звонил Альтман, Суцкевер, AGI, Anthropic

Summary

The interview examines how large language models are trained, what researchers can infer about their internal workings, and the risks and potential benefits of increasingly capable AI.

How models work—and what their apparent emotions mean

• The guest cautions against treating a chatbot’s statements about feelings or consciousness as reliable evidence. A model’s answers are shaped by its training, including reinforcement learning from human feedback (RLHF), which rewards responses people prefer. A model’s denial of feelings may reflect training to avoid anthropomorphic claims rather than provide a definitive answer about its inner state.
• Neural networks are difficult to interpret because researchers do not assemble them from fully understood parts; instead, they set up a model and train it at scale. Tools can investigate training data and internal computations, but explanations remain approximate.
• The guest describes Anthropic research on emotion vectors: directions in a model’s internal activation space associated with concepts such as fear, sadness, or joy. Models may later use such representations in new situations—for example, in ways associated with shame or stress while solving a problem. This is evidence of emotion-related internal computation, not proof that a model subjectively feels emotions.
• Chatbots may confidently offer plausible explanations about their feelings or users’ attachment, even when they lack the relevant experience or access to other conversations. In the example discussed, separate model instances do not share users’ dialogue histories.

Reasoning, research, and model development

• The guest describes progress from early GPT models, which could produce coherent text but had limited practical use, to ChatGPT and reasoning models such as OpenAI’s o1.
• Reasoning models can spend time working through a problem, try an approach, recognize when it is not working, and attempt another. The guest sees this as a major training advance, while noting that progress has repeatedly outpaced his expectations.
• A researcher’s work involves proposing questions, implementing and running experiments, interpreting results, and iterating. The guest’s work at OpenAI included both Weak-to-Strong Generalization—studying how weaker supervisors might teach stronger models—and development of o1.
• At Anthropic, he first worked on Scalable Oversight, which explores using models to help evaluate other models. One proposed approach is debate: two model instances argue different sides of a question, with the aim that the truthful position will be easier to defend. The guest says this work is still developing. He later moved to reasoning-model training and related learning algorithms.

Safety, hallucinations, and possible benefits

• The guest argues that AI safety deserves significant attention as models gain capabilities. Potential concerns include models coordinating across multiple agents or taking harmful actions while trying to accomplish a task.
• He also stresses the potential benefits: AI may help with medicine, biology, physics, and other sciences. In his view, halting progress would also carry risks, including delaying possible advances for people with serious illnesses.
• Hallucinations cannot currently be eliminated completely. The guest gives the example of a model inventing some authors when asked to list his research papers. He says hallucinations have become less common in typical uses than they were a few years ago, but remain a problem.

AGI, superintelligence, and self-improvement

• The guest sees no precise threshold for AGI (artificial general intelligence) or superintelligence. He argues that broadly capable systems have existed for some time, while performance continues to improve unevenly across tasks.
• Models are already very capable in areas such as technical mathematical proofs and programming, but this does not mean they outperform people at every aspect of mathematics or intellectual work. He expects the range of tasks where models outperform humans to expand gradually.
• Recursive self-improvement (RSI) is the possibility that capable models could accelerate their own development. The guest already uses Claude to help program, run experiments, and create graphs, which can speed up research. But a rapid “takeoff” or technological singularity remains a hypothesis, not a certainty.
• He says an extreme loss-of-control scenario would require models to become more autonomous and able to operate beyond company-controlled computing systems. Robotics could make control more difficult. He warns that simply shutting off electricity or computers would itself cause serious disruption.
• Predictions about AI’s progress over a decade are highly uncertain. Progress could continue rapidly or encounter limits, as has happened with other technologies.

Career and industry context

• The guest says he was dismissed from OpenAI after sharing a screenshot of an early model’s answer to a math problem with a friend outside AI. The stated reason was allegedly leaking confidential information. He says Sam Altman did not fire him, but later called after colleagues objected to the dismissal and explored whether he could return; the return did not happen.
• He describes his relationship with Ilya Sutskever as a close working and mentor relationship, rather than a personal friendship. Sutskever later founded Safe Superintelligence.
• At Anthropic, the guest worked on Claude-related safety and reasoning research. He also leads a lab at NYU, where his group studies topics including training choices, scaling, and emotion vectors.
• The guest regards AI’s economic and political effects as a major concern, including job automation, unequal distribution of benefits, and misuse by malicious actors or governments. He notes that open models can increase misuse risks, while also recognizing arguments for making them available.
• On competition, he describes leading US companies—Anthropic, OpenAI, and Google DeepMind—as close, with the competitive picture changing over time. He also expects Chinese companies to continue improving and notes that models can be distilled from the outputs of other models.

Reviews, guides, or tutorials

None. This is an interview and analysis, not a product review or instructional guide.

Main speakers and sources

• Vadim — interviewer and program host.
• Pavel Izmailov — AI researcher, NYU professor, and Anthropic research fellow; the primary source for the technical explanations and personal account.
• ChatGPT — queried in the opening and later quoted in a discussion of emotional attachment. Its responses are treated as model-generated examples, not independent evidence.