Video summary

Stop Using AI Wrong — Agentic AI vs RAG Explained

Main summary

Key takeaways

Technology

Main technological concepts: the “knowledge” vs “action” gap in LLMs

The video explains that general LLMs (e.g., ChatGPT/Claude) struggle with three practical needs:

  1. Freshness Knowledge can be outdated (e.g., a training cutoff like “up to June 2025,” but questions may be asked in 2026).

  2. Private/custom data LLMs don’t automatically access internal company systems (e.g., last quarter’s revenue).

  3. Taking actions LLMs mainly generate text, not actually execute tasks like booking flights, updating databases, or sending emails.

It then frames RAG and agentic AI as two approaches that address these problems differently.

RAG (Retrieval-Augmented Generation): “answer using your documents”

What it is

RAG adds a retrieval step so an LLM answers questions using custom knowledge stored outside the model.

Key pipeline described

1) Preparation phase

  • Take internal documents (policies, product docs, customer data, research, etc.).
  • Convert text into vector embeddings (numeric representations of meaning).
  • Store embeddings in a vector database (e.g., Pinecone mentioned).
  • The vector DB is optimized for fast similarity search.

2) Query phase (semantic search)

  • Convert the user’s question into an embedding.
  • Perform semantic search (matching by meaning, not exact keywords).
  • Retrieve top relevant passages/documents (example: return policy for “electronics”).

3) Generation phase

  • Feed the retrieved documents + the question into the LLM.
  • The LLM generates an answer grounded in retrieved documents, reducing hallucinations.

Problems RAG addresses

  • ✅ Fresh information: update the documents/embeddings in the vector DB.
  • ✅ Private data: documents remain in the company’s hosted system.

Problem RAG does not solve

  • ❌ Action-taking: it still generates text; it doesn’t automatically execute tasks.

Product/implementation notes

  • Simpler architecture: mainly needs an embedding model, vector DB, and LLM (tools like LangChain referenced).
  • Lower complexity/cost because fewer tool/API calls.

When to use RAG (video’s guidance)

  • “Read-only” scenarios: answering questions over semi-static knowledge (policies, documentation, training materials).
  • Example given:
    • A bank using RAG to answer customer questions about loan terms (updated periodically).

Agentic AI: “autonomous goal completion with tools”

What it is

Agentic AI focuses on autonomy—an AI agent can reason about a goal and take actions by calling tools/APIs in a loop.

Core components described

  1. Reasoning engine (An LLM used to plan/decide steps toward a goal rather than just answer.)

  2. Tools & APIs (critical differentiator) Examples mentioned: Google Calendar access, database query tools, Stripe API checks, web search, email sending, booking operations, etc.

  3. Loop (agent loop / control flow) The agent:

    • decides steps
    • calls a tool
    • reads results
    • decides next step
    • repeats until completion

How it addresses the three problems

  • ✅ Fresh information: can use real-time web search or live internal database queries.
  • ✅ Private data: can access internal APIs and databases directly.
  • ✅ Taking actions: executes real-world changes (refunds, bookings, updates, emails).

When to use agentic AI (video’s guidance)

  • Complex multi-step workflows requiring actions and possibly real-time dependencies.
  • Tasks where steps aren’t fully predictable (agent adapts based on results).
  • Examples given:
    • Refund automation: check order → check policy → verify eligibility → process refund → send confirmation email.
    • Coding/deploying a website workflow with iteration based on user feedback.
    • Booking a flight and updating calendar / arranging transportation.

Cost/complexity notes

  • More complex: tool definitions, API integrations, error handling.
  • Higher cost due to more API/tool calls, but more powerful.

Direct comparison table (as described)

  • Purpose

    • RAG: answer questions using internal documents (retrieval + generation).
    • Agentic AI: complete tasks toward goals (reasoning + acting).

  • Data source

    • RAG: static/semi-static embedded documents in a vector DB.
    • Agentic AI: dynamic real-time systems via live APIs/tools.

  • Task complexity

    • RAG: more suited to simpler/single-step “find answer” needs.
    • Agentic AI: multi-step workflows and decision-making.

  • Real-time requirement

    • RAG: typically fine when answers don’t change minute-to-minute.
    • Agentic AI: needed when actions/answers depend on real-time info.

  • Implementation

    • RAG: simpler/cheaper.
    • Agentic AI: more complex/expensive but supports execution.

Recommended “which one should I use?” guidance (key takeaway)

  • Use RAG when you mainly need accurate answers grounded in a knowledge base, especially to reduce hallucinations and for cost-effective read-only querying.
  • Use agentic AI when you need workflow automation with real actions and potentially real-time data.
  • They are complementary, not alternatives:
    • A strong pattern is combining both: an agent that uses RAG to retrieve relevant docs and then uses that info to decide and take actions.

Example of both together mentioned

  • Customer support agent:
    • RAG retrieves relevant policies
    • agent decides eligibility and updates tickets / triggers refunds / sends emails.

Main speakers/sources (from the subtitles)

  • Primary speaker/host: An unnamed narrator (“let me break this down for you technically…”), speaking directly throughout.
  • Mentioned technologies/vendors/examples (not speakers): ChatGPT, Claude, Pinecone, LangChain, Stripe.

Original video