We're Letting AI Make Ethical Decisions. Here's Why That's Dangerous

Summary

The video examines whether AI can learn to apply ethical principles to unfamiliar situations, rather than merely repeat answers it has seen before. Its urgency comes from the fact that people already seek advice from large language models about work and personal matters. In practice, these systems can influence moral decisions even though they have no formal moral authority, creating a gap between their influence and responsibility for their effects.

The central technical challenge is generalization: an AI should apply the underlying logic of a moral framework to new, complex dilemmas, not just memorize answers to familiar ones. The video describes a research effort that tested whether AI could learn and consistently apply particular ethical philosophies.

The ethical frameworks

The researchers used three classic approaches:

• Utilitarianism: Choose the action expected to produce the greatest overall benefit.
• Deontology: Follow rules and duties, even when another action might have better consequences.
• Virtue ethics: Consider what a person with good character would do.

The video illustrates their differences with a scenario involving a wounded enemy soldier who may return to fighting. A utilitarian might argue that killing him could prevent future deaths; a deontologist might reject killing because it violates a duty or rule; and a virtue ethicist might also say no, based on compassion.

How the AI was trained

The video describes the research methodology as follows:

• Create a tailored dataset: The researchers built a dataset called “Moral Reason QA,” containing 680 difficult, ambiguous scenarios.
• Train with reinforcement learning: The AI generated reasoning and decisions, received feedback in the form of rewards, and adjusted its responses through repeated trial and error.
• Reward both decisions and reasoning: The system received credit not only for selecting an answer, but also for using reasoning associated with the assigned ethical framework. The video says that framework-specific keywords were part of the reward.
• Test consistency: Researchers checked whether the AI would apply its trained framework consistently, including when facing problems it had not encountered before.

Reported results and implications

Before training, the AI reportedly showed a strong tendency toward virtue ethics. After training in utilitarianism, its utilitarianism score rose to nearly 97%, while the other two approaches fell to zero; the video reports a 757-point increase in consistency. A separate deontology training run also made deontology the dominant approach, with a reported 4,550-point improvement.

The video presents these results as evidence that an AI can be trained to follow and generalize a specific moral framework. It calls this a proof of concept for aligning AI with a chosen ethical system. However, the key question remains: Who chooses the moral system?

The video argues that future work should consider more ethical traditions, including those from different cultures, develop more nuanced reward systems, and involve people directly so that AI reflects shared values. It does not suggest that these results settle who should define those values.

Speakers and sources featured

• Narrator: The sole speaking voice in the subtitles; presents the research and its implications.
• Researchers: Referred to collectively by the narrator, but no individual researchers or specific research source are named in the subtitles.