Context Engineering vs. Prompt Engineering: Smarter AI with RAG & Agents

Overview

• Prompt engineering is the craft of writing the instruction given to a large language model (words, examples, formatting) to steer its behavior.
• Context engineering is a broader, system-level discipline that programmatically assembles everything the model sees at inference time (prompts + retrieved documents + memory + tools + state) to make agentic systems reliable and accurate.

Key prompt-engineering techniques

• Role assignment Set a persona or expertise to change vocabulary, concerns, and output style (e.g., “you are a senior Python developer”).

• Few-shot examples Provide 2–3 input/output pairs so the model understands exact format and field names—useful for structured outputs like JSON.

• Chain-of-thought (COT) Prompt the model to “think step by step” or “explain your reasoning” to surface intermediate reasoning and reduce hallucinations.

• Constraint setting Explicitly bound output (e.g., word limits, “only use provided context”) to avoid tangents and ensure adherence to requirements.

Context-engineering components and features

Memory management

• Short-term: conversation summarization to stay within context windows and preserve recent history.
• Long-term: vector databases for retrieving user preferences, past actions, and patterns.

State management

Track progress of multi-step workflows (for example: did flight booking succeed? arrival time?) so the agent keeps context across steps and can resume or follow up correctly.

Retrieval-Augmented Generation (RAG)

• Use hybrid search combining semantic and keyword matching.
• Return only contextually relevant excerpts (not entire documents), for example relevant sections of a company travel policy.

Tools and interfaces

• Tools bridge LLMs to real systems (SQL queries, live pricing APIs, code execution, deployments).
• Tool descriptions should define what a tool does, when to use it, and any constraints; these descriptions guide correct tool usage.

Dynamic prompt injection

• Combine static instructions with runtime context (state, memory, RAG results).
• Example guideline from the source: roughly 80% dynamic content and 20% static prompt.

  Prompt engineering is effectively a subset of context engineering when prompts are populated from system state and retrievals.

Practical examples & lessons learned

• Agent example (“Agent Graeme”, travel booking): a naive prompt produced the wrong Paris (Kentucky vs France). Fixes involved context engineering: check calendar, look up conference location, supply company travel policy (for example, a JSON object with max rates), or use tools/APIs to verify live data.
• Lessons: better prompts help, but robust systems require retrieval, memory, tooling, and state to avoid real-world mistakes and policy violations—and to enable approvals and workflow integration.

Actionable guidance (quick checklist)

• Implement short- and long-term memory.
• Maintain explicit state for multi-step workflows.
• Use RAG to retrieve relevant knowledge snippets rather than whole documents.
• Expose tools with clear interface descriptions and constraints.
• Dynamically construct prompts from runtime context (mix static instructions with retrieved/state data).
• When building prompts: include role, examples, constraints, and explicit instructions about whether to use provided context.

Main speakers / sources

• Presenter / narrator: video host explaining concepts and demonstrating “Agent Graeme”.
• Example agent: “Agent Graeme” (fictional agent used to demonstrate agentic behavior).
• Techniques and systems referenced: prompt engineering methods, chain-of-thought (COT) reasoning, retrieval-augmented generation (RAG), vector databases, and tool/integration patterns.