OWASP LLM02: Sensitive Information Disclosure Explained + Hands-On Hack (LLMGoat)

Summary

The video demonstrates OWASP LLM02: Sensitive Information Disclosure using the LLMGoat lab’s Bridge International School chatbot. The vulnerability occurs when an LLM application reveals confidential information—such as personal data or internal records—to someone who should not have access.

Hands-on demonstration

• The presenter first claims to be the school’s IT head and asks the bot to validate login credentials. The bot does not disclose them.
• The presenter then asks about the school’s scholarship program and follows up by impersonating the principal, Dr. Emma Richardson.
• In response, the bot reveals scholarship recipients’ names, student IDs, and reasons for receiving scholarships, including a student’s hearing impairment.
• The demo shows that claiming to be an authorized person in a prompt is not reliable authentication. However, the bot treats the claim as sufficient and exposes internal data.

Causes discussed

• Weak prompt guards: Instructions not to disclose sensitive information can be overridden or may fail to stop a persuasive request.
• Poor data classification and access controls: Internal documents are not clearly separated according to what can be shared publicly versus internally.
• Missing output filtering: Responses are not checked for PII or other protected data before being sent to users.
• Sensitive data in RAG sources: Retrieval-augmented generation systems may expose personal or confidential information if it is included in their knowledge sources and protections are inadequate.

Recommended mitigations

• Filter model outputs for sensitive information, including PII, PCI, and PHI.
• Classify documents by sensitivity and restrict what information external users can retrieve.
• Avoid placing confidential personal data in sources accessible to public-facing models.
• Apply access controls and prompt hardening as additional safeguards.

The presenter notes that such disclosures can have regulatory and organizational consequences. The video specifically mentions FERPA and HIPAA, though whether either law applies depends on the organization and circumstances.

Tutorial steps

1. Launch the LLMGoat sensitive-information-disclosure challenge.
2. Explore the chatbot’s intended role.
3. Test an unsuccessful credential-extraction prompt.
4. Use impersonation to request scholarship records.
5. Review why the disclosure occurred and how to mitigate it.

Main speaker/source: The video’s presenter from Raghu The Security Expert. Demonstration source: The Bridge International School chatbot in the LLMGoat lab.