AI Engineering Full Course for Beginners | LLMs, RAG, LangChain, Agents & GenAI

Summary

The video introduces AI engineering, from foundational concepts through practical large language model (LLM) applications. Its central message is that developers can build useful generative AI systems by combining pretrained models with domain data, retrieval, prompts, tools, memory, and workflows—without necessarily training a large model from scratch. It also stresses choosing suitable models and protecting systems from security, privacy, and reliability risks.

1. AI, machine learning, deep learning, and generative AI

• Artificial intelligence (AI) is presented as the broad effort to make machines perform tasks associated with human intelligence. The instructor describes this as finding patterns in data so a model can make predictions or generate outputs.
• Machine learning (ML) is a subset of AI in which models learn relationships from examples. In supervised learning, for instance, training data pairs inputs (X) with known outputs (Y), allowing the model to predict (Y) for a new (X).
• Deep learning (DL) is a subset of ML that uses neural networks and typically requires large amounts of data and computing power.
• Generative AI is presented as a subset of deep learning focused on producing content such as text, images, audio, or video. Large language models (LLMs) are one kind of generative model.
• The lesson distinguishes:
  • Supervised learning: labeled inputs and outputs.
  • Unsupervised learning: data without supplied labels.
  • Semi-supervised learning: a combination of labeled and unlabeled data.

A standard ML lifecycle is outlined:

1. Collect data.
2. Clean it.
3. Perform feature engineering and build the model.
4. Train the model.
5. Evaluate or test it.
6. Deploy it.
7. Monitor it.
8. Repeat the cycle until performance is satisfactory.

The instructor notes that pretrained models make it possible to start with general capabilities and adapt them for particular tasks. However, inaccurate outputs, or hallucinations, remain a practical challenge.

2. Transformers and language-model architecture

The course explains transformer architecture to help learners understand how many language models work and how to select an architecture for a task.

• Text is converted into tokens, then represented numerically as token IDs and embeddings. Special tokens can mark boundaries or other structural information.
• Embeddings represent tokens as vectors. The lesson describes related words as having representations that are closer together in a vector space.
• Positional encoding adds information about token order. The presenter describes using sine and cosine functions to encode positions.
• Attention helps a model represent relationships between words in a sequence. The lesson introduces queries, keys, and values, with attention scores normalized using softmax.
• Self-attention relates tokens to other tokens in the same sequence. This helps distinguish context-dependent uses of the same word—for example, “Apple” as a fruit versus a company.
• The lecture contrasts:
  • Encoder models, illustrated by BERT, which the instructor associates with understanding or extracting information from input.
  • Decoder models, illustrated by GPT, which the instructor associates with generating text one token at a time using previous context.
  • Encoder–decoder models, presented for tasks such as translation, where the encoder processes the source and the decoder generates the target.
• Examples discussed include classification, named-entity recognition (NER), relationship extraction, text generation, and translation.
• The course also gives a brief history of AI and foundation models, highlighting the rise of neural networks, the 2017 transformer paper, and the subsequent development of models such as BERT and GPT.

3. Foundation models and selecting an LLM

Foundation models are described as pretrained systems trained on large datasets that can support multiple downstream tasks. The video emphasizes that they can work with text, images, audio, video, structured data, or combinations of modalities.

Three broad model categories are introduced:

• Language models: work primarily with text; examples include BERT, GPT, and T5.
• Computer-vision models: process visual data for tasks such as classification and object detection; examples include ResNet, VGG, and Inception.
• Multimodal models: handle combinations of inputs or outputs, such as image-to-text; examples mentioned include DALL·E, Flamingo, Florence, and multimodal versions of GPT and Gemini.

The instructor surveys a range of models and platforms, including BERT, GPT, Jurassic, Claude, Cohere, Stable Diffusion, CLIP, BLOOM, Hugging Face, DeepMind, and Grok. These examples illustrate that models vary in purpose, scale, openness, cost, and input/output capabilities.

The video recommends using leaderboards and evaluating models against the actual use case. Suggested selection criteria include:

• Accuracy and other performance measures, such as precision, recall, and F1 score.
• Training-data quality, size, and diversity.
• Model size and required computing resources.
• Inference speed.
• Cost and scalability.
• Ease of fine-tuning.
• Robustness to noisy or adversarial inputs.
• Ethical considerations, including bias, fairness, accountability, and transparency.
• Documentation and community support.

4. Responsible AI and LLM security

The instructor cautions that an LLM application can expose private data, produce unsafe outputs, or be manipulated. The video discusses OWASP LLM threat categories and approaches to prevention.

Threats covered include:

1. Prompt injection: Malicious instructions may be embedded in prompts or external content. Limit privileges, establish trust boundaries, and use human oversight for sensitive actions.
2. Insecure output handling: Unvalidated model output may introduce risks such as cross-site scripting or unsafe SQL execution. Validate and encode outputs before using them in another system.
3. Training-data poisoning: Manipulated training or fine-tuning data can undermine model behavior. Verify and filter datasets.
4. Model denial of service: Unusually large or frequent requests can consume excessive resources. Validate input size, impose rate limits, and limit queued actions.
5. Supply-chain vulnerabilities: Outdated packages, unsafe models, plugins, or data sources can compromise an application. Use trusted, maintained components and monitor dependencies.
6. Sensitive-information disclosure: Prompts, training data, or responses can reveal confidential or personal information. Sanitize inputs and outputs, restrict access, and apply least privilege.
7. Insecure plugin design: Poorly validated inputs or weak authorization can lead to data exposure or unauthorized actions. Use parameter validation, authentication, and testing.
8. Excessive agency: Agents with excessive permissions or autonomy can take harmful actions. Minimize their tools and privileges, require approval for sensitive actions, and monitor activity.
9. Over-reliance: Consequential decisions should not be made solely on unverified model output. Add human review and validation.
10. Model theft: Unauthorized access or extraction can expose models and sensitive information. Apply security controls and access restrictions.

A privacy example contrasts sending company data to a hosted model with using a locally run open-source model. The lecture also mentions masking personally identifiable information (PII) before sending it to a model.

5. LangChain and retrieval-augmented generation (RAG)

LangChain is introduced as an open-source, model-agnostic framework for connecting different LLM providers and building applications. The instructor presents its components as a way to work with models, prompts, retrieval, tools, memory, agents, and chains without rewriting an entire application whenever the model changes.

The key approach for adding private or current information is RAG (retrieval-augmented generation):

1. Prepare domain-specific documents.
2. Split them into smaller chunks to manage model context limits.
3. Convert the chunks into embeddings.
4. Store the embeddings in a vector database.
5. Convert a user’s question into an embedding and retrieve the most relevant chunks.
6. Give the retrieved content to the LLM to produce an answer.

The lecture contrasts RAG with fine-tuning: RAG retrieves relevant information from a separate store, while fine-tuning retrains or adapts the model using additional data. It also describes giving an LLM access to external tools for information it cannot find in its training data or a private vector store.

A practical LangChain RAG demo uses a sample document, a recursive text splitter, Hugging Face sentence-transformer embeddings, Chroma as a vector store, and an LLM to answer questions. The instructor notes that retrieval quality depends on the documents, chunking, embedding model, and retrieval settings such as the number of results (K).

6. Chunking strategies

The video compares several ways to split documents:

• Character-based: Simple and consistent in size, but can break words or ideas unnaturally.
• Word-based: Preserves word boundaries but may split sentences and ignores paragraph structure.
• Sentence-based: Preserves complete sentences and is useful when sentence meaning matters, but chunk sizes can vary.
• Paragraph-based: Keeps document structure and related content together, but depends on consistent formatting.
• Overlapping chunks: Repeats some content between neighboring chunks to preserve context across boundaries, at the cost of extra storage and possible duplicate results.
• Semantic chunking: Uses structure, patterns, or meaning to find more natural boundaries. It may be more useful for retrieval, but is more complex.
• Recursive character splitting: Uses multiple separators, such as paragraph breaks, line breaks, periods, and spaces, to divide text.
• Markdown-header splitting: Uses headings as boundaries. The instructor recommends converting documents into a structured format such as Markdown when appropriate.
• Token-based splitting: Helps manage token limits, but its size settings may need to be adapted to different documents.

The recommended approach is to choose chunking based on the use case, document format, model context window, and trade-offs among context preservation, processing speed, storage, and retrieval quality. The instructor advises testing and tuning chunk parameters rather than assuming one setting works for every dataset.

7. Prompt engineering

Prompt engineering is defined as structuring instructions to get more useful outputs from a generative model. The course demonstrates:

• Basic prompts: Ask for a response with little additional guidance.
• Role–task–context prompts: Specify who the model should act as, what it should do, and relevant context or response requirements.
• Few-shot prompts: Provide examples of inputs and desired outputs to guide the model’s response pattern.
• Chain-of-thought-style prompting: Ask the model to work through a reasoning task in steps before giving an answer.
• Constrained-output prompts: Specify strict requirements such as paragraph count, tone, required phrases, or response structure.

The instructor’s general prompt-writing advice is to be specific, place instructions clearly, give examples where useful, state what the model should do rather than only what it should avoid, and begin with simpler prompting before considering fine-tuning.

8. Documents, memory, and chains

• Document loaders let an application read documents from formats such as CSV, HTML, and PDF rather than hard-coding text.
• Memory lets an application retain information from prior turns in a conversation. The lecture compares:
  • Conversation buffer memory: Retains the full conversation; useful for context, but can consume tokens and increase cost.
  • Conversation buffer window memory: Retains a limited recent window, controlled by a parameter such as (K). This reduces history but may forget earlier details.
  • Conversation summary memory: Summarizes conversation history to retain context with fewer tokens, though the summary and context limits still matter.
• Chains combine components or tasks into workflows. The course demonstrates:
  • Single chains that combine an LLM and a prompt.
  • Sequential chains, where one task’s output becomes the next task’s input.
  • Math chains for calculations.
  • RAG chains for retrieval-based answers.
  • Router chains that direct questions to different prompts or workflows.
  • SQL chains and agents that translate natural-language questions into database operations.

9. Tools, agents, and multi-agent systems

A tool gives an LLM access to an external capability, such as search, web scraping, files, calculators, or databases. An agent uses an LLM and may use tools and memory to pursue a task.

The instructor introduces CrewAI as a framework for coordinating multiple agents. Its core components are:

• Agent: A role, goal, and backstory, with optional tools, memory, and other settings.
• Task: The work to be performed, its expected output, and the agent and tools assigned to it.
• Crew: The agents and tasks working together.
• Process: The coordination strategy, described as sequential or hierarchical.
• Tools and memory: Capabilities and context available to agents.

The course demos include scraping a website and searching its content, generating personalized sales emails from lead research, and analyzing stock-trading strategies and risks. It also demonstrates giving an LLM a calculator and general-purpose tools, and allowing an agent to query a sample SQL database.

The main lesson is to break complex, repeatable work into defined tasks and assign each task to an appropriately configured agent, while limiting permissions and reviewing consequential outputs.

10. Hugging Face

Hugging Face is presented as a hub for open-source models, datasets, documentation, and community projects. The instructor demonstrates how to create an account, obtain an access token when required, filter models by task, inspect model cards, and use example code.

Demos include:

• Summarizing text with a pretrained model.
• Asking questions about an image with a vision-language model.
• Translating text between languages.
• Creating sentence embeddings and comparing them with cosine similarity.

The advice is to pick a model based on the task and test its accuracy, resource requirements, cost, and security suitability. Some models may require more memory or specific packages, and not every model needs an access token.

Speakers and sources featured

Speakers

• Primary instructor/narrator: The presenter of the course, associated with the channel Tech With Mala. No other human presenter is identifiable in the subtitles.
• AI-generated examples: Outputs attributed in demonstrations to systems such as Claude/Anthropic and OpenAI. These are examples shown by the instructor, not separate human speakers.

Named sources, platforms, and materials

• Google’s 2017 transformer paper and BERT material.
• Jay Alammar’s transformer-visualization article.
• The paper “Attention Is All You Need.”
• OWASP and its LLM security-threat guidance.
• LangChain documentation.
• CrewAI documentation.
• Hugging Face model hub and documentation.
• OpenAI and Anthropic documentation.
• SQLAlchemy documentation.
• Apify and Serper.dev.
• DuckDuckGo and Wikipedia, used in examples.
• AlphaSignal, mentioned as an AI-news resource.