Integrating LLM with Your Application | LLM to GenAI Application | FDE Full Course #3

Summary

The video explains how to connect an application to a large language model (LLM) and distinguishes a raw LLM model from a complete generative AI application. An LLM provides text-generation capabilities, while products such as ChatGPT combine a model with other components—such as safety checks, search, calculators, weather APIs, and code execution—to offer broader functionality.

A direct API request to a model does not automatically include the tools or persistent conversation history available in a hosted chat product. For example, a model without tools may not know the current weather, may make mistakes on arithmetic or counting tasks, and may forget information from a previous API request. Applications can address these limitations by connecting models to external tools and including relevant conversation context in requests.

Concepts and Practical Walkthrough

• LLM tools: A model can identify when a task may require a tool, but the application must provide the tool and translate the model’s request into a format the tool can use. The tool’s result can then be returned to the model to generate a natural-language answer.
• API access: The demonstration uses Postman to send a request to an OpenAI API endpoint. It covers selecting a model, providing input, and authenticating with a secret API key sent as a bearer token. The response includes generated text and metadata, such as input and output token counts.
• Tokens, cost, and model choice: Usage is based on both input and output tokens. The instructor compares model prices and recommends starting with a less expensive model for practice. Costs vary by model, and free models may have limitations.
• Spring Boot integration: The tutorial builds a support-ticket summarizer API using Spring AI’s ChatClient. A controller accepts ticket text, a service asks the model to summarize it in two lines, and the application returns the generated summary. The example ticket describes a restaurant order that arrived incorrectly after a long wait.
• Prompt boundaries and context: A user could submit an unrelated question—such as “What is 2 + 2?”—to an application intended for food-delivery support. The application should constrain responses to its intended domain. The video also notes that API calls do not retain previous messages automatically, so the model cannot remember a name provided in a separate request.

Tutorial Topics

1. How a hosted AI chat application differs from a direct LLM API call
2. Why external tools are needed for live information and reliable task execution
3. How to make an authenticated LLM request using Postman
4. How to inspect the response and token usage
5. How to connect a Spring Boot application to an LLM with Spring AI
6. How to build a basic support-ticket summarization endpoint

The instructor previews system prompts, conversation roles, and context management as topics for a later lesson.

Main speaker/source: Aditya Tandon, presenting for Coder Army