RAG expliqué simplement : la technique derrière les IA qui marchent

Main ideas / lessons

• LLMs (ChatGPT/Claude/Gemini) are trained on public data, but the most valuable company data is typically private (emails, internal documents, notes). Without special techniques, the model can’t access or use that private content.
• RAG (Retrieval-Augmented Generation) enables an LLM to answer questions using your own documents by:
  1. Indexing documents into a vector database
  2. Retrieving the most relevant chunks for a user question
  3. Generating an answer grounded in those retrieved chunks
• Basic/vanilla RAG works technically, but often fails in production because of:
  • Poorly worded or ambiguous questions
  • Retrieved chunks that are not relevant
  • Hallucinations or confusion caused by irrelevant context
• Modern “corrective/adaptive” RAG improves robustness by adding self-checking and iterative repair (e.g., reformulate, re-retrieve, verify, and regenerate).

Note: The subtitles contain repeated/garbled terms (e.g., “raging/ragu/rac”). The consistent topic is RAG.

---

Methodology / instructions (detailed)

Part 1 — Transform documents into a searchable “map” (Indexing)

1. Chunking (cut documents into pieces)

   • Split a set of documents (PDFs, Word files, notes, raw files) into smaller text chunks.
   • Why: embedding models have a context/window size limit (examples given: ~500 words, sometimes up to ~2000).
   • Choosing chunk size (trade-off):
     • Too short (e.g., ~50 words): loses context; chunks may be unclear or isolated.
     • Too long (e.g., ~5000 words): multiple ideas get mixed; embeddings become “blurry,” harming retrieval precision.
     • Typical sweet spot mentioned: ~500–1000 words
   • Where to cut:
     • Avoid cutting mid-sentence/mid-idea
     • Prefer natural boundaries: end of paragraph/section/chapter
   • Add overlap between chunks:
     • Example ranges:
       • Piece 1: 0–500
       • Piece 2: 450–950
       • Piece 3: 900–1400
     • Goal: if an idea spans boundaries, it appears in both neighboring chunks.
   • Advanced option (mentioned, not detailed): semantic chunking
     • Split based on topic changes rather than only length.

2. Embedding (convert chunks into vectors)

   • Each chunk becomes an embedding: a list of numbers representing the chunk’s meaning.
   • Embedding model vs LLM:
     • LLM: predicts/generates text
     • Embedding model: outputs vectors only (no text generation)
   • Training intuition:
     • Similar-topic texts → similar vectors
     • Unrelated texts → distant vectors
   • Examples of embedding models mentioned:
     • OpenAI text-embedding-3
     • BGE (open-source)
   • Vector size intuition:
     • “Approximately 1500 numbers” (dimensions)
     • Think of each dimension as a “slider” capturing aspects of meaning (topic, emotions, technicality, etc.).

3. Storage (vector database)

   • Store embeddings in a vector database designed for similarity search (nearest-neighbor lookups).
   • Examples named:
     • Pinecone
     • Chroma
     • Weaviate
   • Outcome:
     • You create a semantic map where related chunks are close together in vector space.

---

Part 2 — Find relevant chunks and answer (Retrieval + Generation)

1. Vectorize the user question

   • Use the same embedding model used for document embeddings.
   • Golden rule:
     • One embedding model for the entire system (start to finish)
   • Why:
     • Different embedding models create different vector “worlds” (different coordinate systems), making vectors not comparable.

2. Nearest-neighbor search (KNN)

   • Retrieve the top k closest chunks to the question vector (example k: 3 or 5).
   • Similarity metric:
     • Cosine similarity (angle-based similarity)
   • Interpretation:
     • Near 1 → very similar
     • Near 0 → irrelevant
     • Near -1 → opposite/contradictory
   • Mechanism:
     • Compute cosine similarity between question vector and stored chunk vectors
     • Rank and keep top-k

3. Prompting the LLM using retrieved context

   • Build a prompt like:
     • “Answer the following question based solely on the context below.”
   • Include:
     • Retrieved chunks under Context:
     • The user question under Question:
   • The LLM generates an answer rooted in retrieved chunks (not purely from general knowledge).

Vanilla RAG flow (recap)

• User asks question
• Embed question
• Retrieve top 5 closest chunks (KNN)
• Put chunks into prompt
• Send prompt to LLM
• Return answer

---

Part 3 — Why basic RAG fails, and how to make it robust

Weakness 1: Poorly worded / ambiguous questions

• Failure cause:
  • Vague questions lead similarity search to retrieve the wrong chunks (“garbage in, garbage out”).
• Solutions:
  • Query translation / Multi-query
    • Ask the LLM to rewrite the question in multiple ways (“from different angles”)
    • Run retrieval for each reformulation
    • Combine the results
  • HyDE (Hypothetical Document Embeddings) (described)
    • Instead of embedding the real question directly:
      1. Ask the LLM to invent a hypothetical answer/document
      2. Use that invented text as the search key
      3. Retrieve real documents using embedding similarity
    • Why it helps:
      • The fake answer tends to share format/vocabulary/length with real policy documents, improving cosine-similarity matching.

Weakness 2: Inaccurate retrieval / irrelevant chunks polluting the answer

• Failure cause:
  • Even top-k can include irrelevant chunks.
  • The LLM may treat them as context and steer the response incorrectly.
• Solution: Reranking
  • Use a two-stage ranking strategy:
    1. Candidate retrieval (fast, crude)
       • Use embeddings + cosine similarity to fetch many candidates (example: 50)
    2. Cross-encoder reranking (slow, precise)
       • A cross encoder reads question + candidate chunk together (word-by-word)
       • Produces a more accurate relevance score
       • Keep the best 5
  • Key trade-off:
    • Cosine similarity is fast but shallow
    • Cross encoders are more accurate but too slow to apply across millions directly

Weakness 3 (implied): Hallucination and lack of self-correction

• Problem:
  • The system can still produce answers not grounded in retrieved evidence.
• Solution: Corrective / Adaptive RAG (self-checking loop)
  • Core idea:
    • After each stage, verify whether retrieval/context is relevant and whether the final answer is grounded.
  • If not grounded/relevant:
    • Rephrase the question and retry retrieval
    • If still failing:
      • Search the web for the generated answer
    • If hallucination is detected:
      • Regenerate (loop again)
  • Outcome:
    • The pipeline becomes iterative and self-correcting, not strictly linear.

Three key points to remember (as stated)

1. RAG has three stages:
   • Index documents in vector form
   • Retrieve closest pieces
   • Generate an answer anchored in them
2. Basic RAG often fails due to:
   • Poorly formulated questions
   • Inaccurate retrieval
   • Hallucination
3. Modern RAG adds robustness via:
   • Reformulation (e.g., HyDE)
   • Reranking with cross encoder
   • Verification/self-check at each stage (adaptive/corrective loops)

---

Speakers / sources featured (as mentioned in subtitles)

• Host / narrator (unnamed)
  • Mentions consulting several sources and explains the material simply
• Lance Martin, engineer at Longchain
  • Described as a leading expert on the topic
• Project AI
  • Mentioned as the channel to subscribe to
• Referenced model providers / systems (not individual people):
  • OpenAI (e.g., “Text embedding 3”)
  • Google (implied via “Gemini”)
  • Anthropic (implied via “Claude”)
  • Vector database examples: Pinecone, Chroma, Weaviate