Structured Output in LangChain | Generative AI using LangChain | Video 5 | CampusX

Main ideas / lessons

• Goal of the video: show how to make an LLM produce structured output so it can integrate with databases, APIs, and agent tools (i.e., other machines/systems), not just talk to humans.
• Recap from prior video: LangChain prompts are inputs you give to an LLM.
• Current focus: processing the output of an LLM using structured output.

Unstructured vs structured output

• Unstructured output (default LLM behavior):

  • LLMs typically return plain text.

  • Example idea: asking for “capital of India” returns text like: “New Delhi is the capital of India.”

  • This “no fixed format” nature is why it’s difficult to directly connect LLM output to programs.

• Structured output:

  • The same kind of content is returned in a well-defined data format (e.g., JSON).
  • Example concept: a “one-day travel event for Paris” can be returned as JSON with:
    • multiple items
    • each with fields like time and activity (represented as dictionaries/objects)

• Definition / benefit:

  • Structured output = language models return responses in a well-defined data format.
  • Key benefit: output becomes easy to programmatically parse, validate, store, and send to other systems.

Use cases (3 primary ones)

1. Data extraction

   • When you need to store LLM-derived information in a database.
   • Example workflow (job portal):
     • user uploads resume
     • extract resume text
     • LLM outputs extracted fields (candidate name, last company, marks, etc.) in JSON
     • code inserts JSON fields into a database

2. Building APIs / enriching review data

   • Example: long user reviews (unstructured text).
   • LLM produces structured fields such as:
     • topic areas (e.g., battery/display/processor)
     • pros
     • cons
     • overall sentiment
   • Then the system exposes this via an API framework (e.g., Flask/FastAPI), so others can consume the structured results.

3. Agents (tools require structured inputs)

   • Agents are “chatbots on steroids” that can perform tasks using tools.
   • Tools often require non-text structured arguments (e.g., calculator needs numbers).
   • Structured output helps an agent:
     • interpret user intent
     • convert it into the exact structured arguments required by tools
     • send those arguments into the tool

Method in LangChain: handling different model capabilities

• There are two types of LLMs discussed:

  1. Models that can generate structured output by default
     • Example mentioned: OpenAI GPT models
  2. Models that cannot generate structured output by default
     • For these, LangChain uses output parsels (covered in the next video).

• In this video, focus is on LangChain’s withStructuredOutput approach for capable models.

Detailed methodology / “how to do it” in LangChain

Overall flow (conceptual)

• Instead of directly invoking the model and parsing free-form text:
  • wrap the model with withStructuredOutput
  • provide a schema describing the desired structured fields
  • invoke the structured model
  • receive output in the target structured format (commonly JSON-like dictionaries/objects)

Where schema comes from: 3 schema formats

1) Typed Dictionary (Python type hints; minimal validation)

• What it is:

  • A Python TypedDict defines expected keys and value types (for editor/type-hint guidance).

• Key properties:

  • Helps communicate structure to developers/editors.
  • Does not guarantee runtime validation (the model can still return incorrect types/format).

• Video example: phone review

  • Schema fields:
    • summary: string
    • sentiment: string
  • Model wrapped with withStructuredOutput(schema=TypedDictClass)
  • Result returned as a structured dictionary, and fields like result.summary / result.sentiment are extracted.

• Extensions shown:

  • Annotations / descriptions for fields to guide the LLM more explicitly.
  • Lists for multiple items:
    • themes: List[str]
    • pros: Optional[List[str]]
    • cons: Optional[List[str]]
  • Optional fields
    • fields can be absent if not present in the source text.
  • Literal for constrained outputs
    • e.g., sentiment limited to allowed values like "positive" or "negative" (the video suggests returning abbreviated forms like POS/NG depending on schema setup).

• Limitation explicitly stated:

  • No true validation—LLM might violate the intended types/conditions.

2) Pydantic (adds runtime validation + parsing)

• What it is:

  • Pydantic is a data validation and parsing library.
  • Common in FastAPI contexts.

• Key features demonstrated:

  • Validation with errors if incorrect types are provided (e.g., age must be string; providing int raises error).
  • Default values
  • Optional fields (missing values become None)
  • Implicit type coercion
    • if you expect int but receive numeric string, Pydantic may convert it.
  • Built-in types
    • e.g., validating email via EmailStr-like behavior.
  • Constraints using field()
    • numeric ranges (e.g., CGPA between 0 and 10)
    • descriptions
    • regular expressions (for patterns like phone numbers)

• Video example: smartphone review schema

  • Implement the schema as a Pydantic model, including:
    • themes: List[str] (with field description)
    • summary: str
    • sentiment: Literal["positive","negative"] (example)
    • pros: Optional[List[str]]
    • cons: Optional[List[str]]
    • optional name: Optional[str]
  • Use withStructuredOutput with the Pydantic schema.
  • Result is a Pydantic object:
    • can access fields directly, or convert to:
      • dictionary via .dict()
      • JSON via .model_dump_json() (or similar in-code method names)

3) JSON Schema (cross-language compatibility)

• When to use:

  • If the project uses multiple languages (Python backend + JavaScript frontend, etc.).

• Why it helps:

  • JSON Schema is universal, so other languages can understand the schema.

• How it’s structured (as shown):

  • schema has:
    • title
    • description (optional)
    • type (e.g., object)
    • properties (field definitions)
    • required (list of mandatory fields)

• Video example: smartphone review

  • Schema includes:
    • themes, summary, sentiment as required
    • pros, cons as optional (represented like string or null / array or null patterns)
  • The structured output produced is parsed into a Python dictionary-like object.

Choosing between the three

• TypedDict
  • Best when everything is in Python and you mainly want type hints (rare in real-world validation-heavy use).
• Pydantic
  • Go-to choice in Python-heavy systems when you need validation, defaults, and coercion.
• JSON Schema
  • Best when schemas must be shared across languages and systems.

Structured output modes: JSON mode vs function calling

• In LangChain’s withStructuredOutput, a configuration option includes:

  1. json_mode
     • get structured output as JSON (common/general case)
  2. function_calling
     • structured output is formatted to support calling a function (useful for agents/tools)

• Rule of thumb from the video:

  • For OpenAI models, prefer function calling (often default).
  • For other models (e.g., mentioned: Cohere/Gemini), use JSON mode when supported.

• Models without support:

  • If a model doesn’t support either JSON mode or function calling structured output, you must use output parcels (next lecture).

Speakers / sources featured

• Speaker: Nitish (host/presenter; stated at the beginning: “my name is Nitish”)
• Sources / technologies referenced:
  • LangChain
  • OpenAI (GPT models mentioned)
  • Pydantic
  • FastAPI
  • Flask / FastAPI (API examples)
  • TinyLlama via Hugging Face Transformers (used as example of a model that doesn’t support structured output modes in the demo)
  • Hugging Face (mentioned for the TinyLlama model example)
  • Agents concept (as discussed in the referenced “previous video”)