Lesson 3A: What is generative AI? (Deep Dive) | AI Fluency: Framework & Foundations Course

Main ideas / concepts

• Generative AI definition

  • Generative AI systems can create new content (e.g., text, responses) rather than only analyzing existing data.
  • Contrast with traditional AI:
    • Traditional AI: classify/categorize (e.g., spam vs. not spam).
    • Generative AI: generate something new (e.g., write a brand-new email).

• Large Language Models (LLMs) as a major type of generative AI

  • LLMs (referred to as “Anthropic’s Claude/Claude-like” cloud models in the subtitles) are language models that:
    • Are trained to predict and generate human language.
    • Are large due to containing billions of parameters (learned numerical values that govern how the model processes information).

How modern generative AI works “under the hood”

1) Pre-training (initial training)

• What happens

  • The model is trained on billions of text examples.
  • It learns statistical relationships among:
    • words, phrases, and concepts (language patterns and meaning in context).

• Mechanism

  • It is shown text and tasked to predict the next token/word.
  • Through many iterations, it refines predictions to produce coherent, meaningful language.

• Outcome

  • The model effectively builds a large internal “map” of language and knowledge based on learned patterns.

2) Fine-tuning (instruction-following + safety)

• What happens

  • After pre-training, models are further trained to:
    • follow instructions
    • give helpful responses
    • avoid harmful content

• Common approaches mentioned

  • Human feedback to improve model behavior.
  • Reinforcement learning using:
    • rewards and penalties to shape the model toward being more helpful, honest, and harmless.

3) Deployment / interaction (prompting)

• How users get outputs

  • Users interact by sending a prompt (text).
  • The model continues/generates new text based on learned patterns from training.

• Clarification

  • It does not simply retrieve pre-written answers from a database.
  • Instead, it generates output that statistically follows from the prompt and context.

4) Context window (limit on what can be used at once)

• Concept

  • LLMs have a context window, a maximum amount of text they can consider at one time.

• What it includes

  • The prompt, the model’s prior responses, and other shared conversation info.

• Implication

  • The model can’t access beyond the current context window unless using specialized tools (e.g., web search).
  • Context windows are growing over time, but they are still not unlimited.

Why generative AI became powerful now (the “three crucial developments”)

The subtitles attribute today’s capabilities to three developments that aligned:

1. Algorithmic/architectural breakthroughs

   • Transformer architecture (2017) is described as a key game-changer.
   • Transformers effectively process text sequences while maintaining relationships between words over long passages—critical for language understanding.

2. Explosion of digital data

   • LLMs learn from large, diverse text sources such as:
     • websites
     • code repositories
     • other text representing human knowledge and communication

3. Massive increases in computational power

   • Specialized hardware:
     • GPUs (graphics processing units)
     • TPUs (tensor processing units)
   • Distributed computing clusters enable training at scales otherwise infeasible.

Scaling laws and emerging capabilities

• Scaling laws

  • As models get larger, train on more data, and use more compute, performance improves in predictable ways.

• Emergent behaviors

  • As models scale, new capabilities can appear—abilities not explicitly programmed in—such as:
    • step-by-step reasoning
    • adapting to new tasks with minimal instruction

Methodology / instructional list (implicit pipeline presented as steps)

1. Step 1: Pre-train

   • Train the LLM on massive text corpora.
   • Use a next-token prediction objective.
   • Learn language/statistical relationships through repeated prediction refinement.

2. Step 2: Fine-tune

   • Train the model to follow instructions and be helpful.
   • Reduce harmful outputs using:
     • human feedback and/or
     • reinforcement learning with rewards/penalties.

3. Step 3: Deploy for interaction

   • Provide a prompt to the deployed model.
   • Generate new text by continuation based on learned patterns.

4. Step 4: Manage context limits

   • Ensure relevant information stays within the context window.
   • Use specialized tools (e.g., web search) if information is outside the context window.

Speakers / sources featured

• Drew Bent (speaker; described as teacher, programmer, and member of technical staff at Anthropic/Enthropic as written in subtitles)
• Anthropic / Claude (referenced source/model name)