Why Large Language Models Hallucinate

Summary

The video explains what large language model (LLM) hallucinations are, why they occur, and how users can reduce their likelihood.

What hallucinations are

An LLM hallucination is an output that departs from facts or the logic of the conversation. Hallucinations can range from small inconsistencies to entirely fabricated claims.

The presenter illustrates this with three false statements: confusing the Earth–Moon distance with the distance to Mars, attributing the presenter’s brother’s airline job to the presenter, and an early Bard response incorrectly crediting the James Webb Space Telescope with the first image of an exoplanet.

The video groups hallucinations into several types:

• Sentence contradictions: A response contradicts something it said earlier.
• Prompt contradictions: A response conflicts with the user’s instructions—for example, producing a negative restaurant review when asked for a positive one.
• Factual errors: The model states something incorrect as though it were true.
• Irrelevant or nonsensical information: The response includes details that do not meaningfully belong in the answer.

Why hallucinations happen

The video describes several common causes:

• Data quality and coverage: Training material may contain errors, bias, noise, or inconsistencies. It may also fail to cover every topic the model is asked about. A model can generalize from its training without being able to verify whether a claim is accurate or relevant.
• Generation methods and objectives: Text-generation techniques involve trade-offs, such as fluency versus diversity, coherence versus creativity, or accuracy versus novelty. These trade-offs can favor generic, likely wording over a less common but more precise answer.
• Input context: Unclear, incomplete, inconsistent, or contradictory prompts can misdirect the model. The same question may require a different answer depending on the context—for example, whether a discussion is about real cats or the Garfield cartoon.

The presenter notes that the way LLMs produce answers is not fully transparent, even to their engineers, and says that improved reasoning capabilities tend to reduce hallucinations.

Ways to reduce hallucinations

• Write clear, specific prompts. Explain what information you want and what the answer should cover. A focused request for a summary of World War II’s major events, key countries, and causes gives the model more guidance than simply asking, “What happened in World War Two?”
• Use generation settings thoughtfully. The temperature setting controls randomness. Lower temperature generally produces more conservative, focused answers; higher temperature encourages more varied and creative responses, but can also increase the opportunity for hallucinations.
• Use multi-shot prompting. Give the model several examples of the kind of response or format you want. Examples can clarify the task and are especially useful for outputs such as code, poetry, or answers in a specified style.
• Supply the relevant context. Make clear what the conversation is about and whether the response should follow a particular context, such as an academic essay or a creative-writing exercise.

Overall, the video’s message is that hallucinations are a known limitation of LLMs. Understanding their causes and providing better prompts, context, examples, and settings can help reduce them.

Speakers and sources mentioned

• Speaker: The video’s presenter, identified in the video context as Martin Keen; no other speakers are featured.
• LLM products or systems mentioned: ChatGPT, Bing Chat, and Google Bard.
• Other named references and examples: IBM, Wikipedia, Reddit, the James Webb Space Telescope, the Garfield cartoon strip, Barack Obama, Paris, and World War II.