Deploying AI Models with Hugging Face – Hands-On Course

Summary of the tutorial (Hugging Face + Transformers/Diffusers + Gradio, end-to-end)

The video presents a hands-on, end-to-end walkthrough of the Hugging Face ecosystem: how to find models, run them with Transformers/Diffusers, understand core model mechanics (especially GPT-2 tokenization + generation), evaluate generation/sampling strategies, perform common NLP tasks (sentiment, NER, QA, translation), process audio (classification, ASR, text-to-speech), generate images (Diffusers/DDPM + Stable Diffusion XL), generate video (Stable Video Diffusion + image-to-video XL models), and finally deploy interactive ML apps using Gradio and Hugging Face Spaces.

---

1) Hugging Face ecosystem overview (product workflow)

Hugging Face is positioned as an “open platform” connecting:

• Models (ML model weights + model cards)
• Datasets
• Spaces (interactive demo front-ends, often Gradio)

Workflow demonstrated:

1. Go to Models
2. Choose a task (e.g., text generation)
3. Open the model card
4. Use the recommended code snippets (often via Transformers pipelines)

---

2) Transformers: Text generation with GPT-2 (model mechanics + tokenization + next-token prediction)

Using GPT-2 via pipeline (high-level helper)

• Loads GPT-2 using pipeline("text-generation", model="openai-community/gpt2")-style usage.
• Key feature: the pipeline handles tokenization automatically.
• Output is returned as a list of dictionaries; typically you extract generated_text.

Faster/low-level approach: AutoTokenizer + AutoModel

Uses:

• AutoTokenizer.from_pretrained(...)
• AutoModelForCausalLM.from_pretrained(...)

In this approach, the script must explicitly:

• tokenize text into input IDs
• decode generated token IDs back to readable text

Tokenization analysis (core concept)

• Shows that words are not necessarily single tokens due to subword tokenization.
• Demonstrates:
  • tokenizer(...) returning input IDs (optionally as PyTorch tensors with return_tensors="pt")
  • tokenizer.decode(...) mapping IDs back to subword pieces/text

Conceptual pipeline described:

• tokenization → ids → embeddings → positional encoding → transformer → generation

Next-token generation with logits + argmax

Manual next-token prediction:

1. Run the model to obtain logits
2. Select the token with highest probability using argmax
3. Decode the token ID and append it to the prompt

---

3) Sampling strategies for generation (analysis + tutorial-style implementation)

The tutorial builds and compares strategies for choosing the next token from the vocabulary distribution.

(A) Greedy decoding

• Choose the token with maximum probability (argmax).
• Deterministic, often less diverse.

(B) Top-k sampling

• Keep only the K most likely tokens and discard the rest.
• Normalize kept logits with softmax, then sample from that filtered set.
• Varying K changes diversity.

(C) Top-p (nucleus) sampling

• Keep the smallest set of tokens whose cumulative probability ≥ P.
• Filter others to -inf, apply softmax, then sample.
• Can produce different choices than top-k for the same prompt.

(D) Temperature sampling

• Adjust randomness by scaling logits:
  • higher temperature → flatter distribution → more creativity/diversity
  • lower temperature → sharper distribution → more deterministic output
• Demonstrated by comparing outputs under different temperatures.

(E) Random sampling (softmax-only)

• Sample directly from the full softmax distribution.
• Most stochastic; output varies on each run.

Token “confidence” visualization

• Apply softmax to logits, then use topk to retrieve top tokens and probabilities.
• Decode and print candidate tokens with confidence percentages.

---

4) Transformers: NLP tasks via pipelines (sentiment, NER, QA, translation)

Sentiment analysis (IMDb)

• Uses dataset: StanfordNLP/IMDb
• Runs a sentiment classifier pipeline (default often distilBERT for SST).
• Builds a scoring function that:
  • limits text length (mentions max ~512)
  • returns predicted label (“positive/negative”)
• Adds predictions as a new column and compares against ground truth.

Domain-specific sentiment: FinBERT (financial)

• Loads FinBERT (fine-tuned for financial sentiment).
• Demonstrates:
  • single sentence inference
  • batch inference for multiple sentences
• Emphasizes that domain tuning improves specialized performance.

Named Entity Recognition (NER)

• Uses a default Hugging Face NER model (token classification; BERT-large case fine-tuned).
• Pipeline returns a list of dicts with entities and tags, such as:
  • organization (org)
  • location (countries/states)
  • etc.

Question Answering (extractive QA)

• Uses a default QA model (distilled SQuAD-style).
• Workflow:
  • provide context + question
  • model returns answer + score

Note: The tutorial mentions “RAG-like” behavior conceptually, but the shown implementation corresponds to standard pipeline QA using provided context.

Machine translation

• Uses translation pipeline (shown: Google T5 for English → French).
• Demonstrates translating phrases and extracting translation_text.
• Mentions browsing MT models under the machine translation task.

---

5) Audio processing with Transformers (audio modality)

Audio classification (speech categories)

• Loads an audio classification pipeline with:
  • AutoFeatureExtractor
  • AutoModelForAudioClassification
• Demonstrates:
  • reading audio with librosa
  • feature extraction into tensors
  • model inference and argmax over logits
  • mapping predicted IDs to labels via model.config.id2label

Automatic Speech Recognition (ASR)

• Uses pipeline("automatic-speech-recognition")
• Passes an audio file and returns transcription text.

Text-to-speech (TTS)

• Uses pipeline("text-to-audio")
• Returns:
  • audio array
  • sampling rate
• Visualizes the waveform and plays audio with IPython display.
• Mentions model selection via Hugging Face tasks (ASR/TTS models).

Saving generated audio

• Converts generated numpy arrays into an audio file format (e.g., MP3) using pydub.AudioSegment.

---

6) Images with Diffusers (generation + DDPM internals)

Image preprocessing

• Demonstrates image handling with:
  • PIL + NumPy + matplotlib
  • extracting RGB channels
  • converting to grayscale
  • resizing via OpenCV while maintaining aspect ratio (scale factors)

DDPM: denoising diffusion probabilistic model (faces)

• Uses DDPMPipeline.from_pretrained("google/ddpm-...")
• Generates by:
  • starting from noise
  • progressively denoising across num_inference_steps

Manual internals shown:

• DDPMScheduler and UNet2DModel
• create a random noise tensor
• loop across timesteps:
  • use UNet to predict residual/noise component
  • update via scheduler step
• Discusses CPU vs CUDA placement (including avoiding CUDA→numpy conversion issues)

Prompted generation: Stable Diffusion XL (text-to-image)

• Uses diffusers.DiffusionPipeline.from_pretrained(...) with:
  • stabilityai/stable-diffusion-xl-base-1.0
• Describes the conceptual flow:
  • prompt → text encoder embeddings
  • diffusion refinement → final image
• Mentions optional concepts like refiner/upscaling from the model card.
• Generates example images from natural language prompts.

---

7) Video generation with Diffusers (image-to-video + prompt-to-video)

Stable Video Diffusion (image → video)

Uses StableVideoDiffusionPipeline.from_pretrained(...).

Key implementation details:

• set torch_dtype=torch.float16 (FP16) to avoid OOM
• enable pipe.enable_model_cpu_offload() for memory management
• load an image via load_image
• set generator seed for reproducibility
• call pipeline with image=... and decode_chunk_size=8
• access frames[0] then export via export_to_video(...)

Performance emphasis:

• GPU greatly speeds up inference (CPU can take ~40 minutes per run)

I2VGen-XL (image + prompt → video)

• Uses I2VGenXLModelPipeline (image-to-video XL)
• Includes:
  • repo id stored in a variable
  • FP16 + CPU offload to prevent OOM
  • prompt=..., image=..., num_frames=..., generator=...
• Demonstrates exporting frames to video and qualitative results (e.g., animated sea, bouncing characters).

Model selection in Hugging Face

• Notes browsing Hugging Face video tasks and selecting models using downloads/likes.
• Mentions task routing that often shifts users from Transformers → Diffusers.

---

8) Gradio: building interactive GUIs (tutorial + deployment patterns)

Basic interface and components

Gradio is introduced as a way to build interfaces quickly without writing custom front-end HTML/JS.

Demonstrated components include:

• number inputs/outputs
• text input/output
• slider, dropdown menu
• images
• JSON output via gradio.JSON
• label output via gradio.Label
• multi-output apps (e.g., image + status text)
• themes (gr.themes.* like glass/soft/monochrome)
• layout with gr.Blocks, rows/columns, scaling
• tabs and accordion elements
• CSS injection using gradio.Css

Event handling (core feature)

• Uses:
  • .change(...) listeners (trigger on input changes)
  • .click(...) listeners (trigger on button clicks)
• Demonstrates responsiveness differences when listeners are attached to different sliders.

Errors/validation

• Demonstrates raising:
  • gradio.Error for invalid inputs
• Also shows an alternative:
  • using warnings instead of hard errors

---

9) Integrating Hugging Face models into Gradio apps

Image classification app (ResNet)

• Loads:
  • AutoImageProcessor
  • AutoModelForImageClassification (ResNet-18)
• Process:
  • preprocess image → model logits
  • argmax → map using id2label
• Gradio app accepts an image and returns a class label.

Sentiment analysis app

• Wraps a Transformers sentiment pipeline in Gradio:
  • gradio.Textbox input
  • outputs: predicted label + confidence score

---

10) Hugging Face Spaces deployment (end-to-end shipping a demo)

Spaces concept

• Spaces are Git repositories hosting interactive ML apps/demos.
• Features mentioned:
  • easy deployment
  • interactive demos
  • version control
  • hardware choices
  • community/flexibility

Creating a Gradio Space

Steps shown:

1. “New Space”
2. choose SDK = Gradio
3. choose hardware (CPU basic in example)
4. set license/description
5. create repository

Uploading a custom project (CLI workflow)

Shows a project structure with:

• app.py (Gradio app)
• model.py (model architecture instantiation)
• requirements.txt
• model weight files (.pth/.pt)

Also covers:

• cloning the Space repo
• copying project files into it

Large file handling

• Uses Git LFS to upload model weights larger than typical limits.
• Includes setup/commands for installing/configuring LFS and tracking .pth files.

Result: deployed interactive diffusion-number demo

• Deploys a diffusion model generating MNIST-like digits from text prompts.
• UI accepts a number as text input and outputs:
  • generated digit images
  • generation time
• Mentions additional Spaces under the author’s account (e.g., a food classifier).

---

Main speakers / sources (as inferred from subtitles)

• Primary speaker: Not explicitly named in the subtitles (course instructor/presenter).
• Core sources/tools referenced:
  • Hugging Face Transformers (pipelines, tokenization, NLP tasks)
  • Hugging Face Diffusers (DDPM, Stable Diffusion XL, stable video diffusion, image-to-video pipelines)
  • Gradio (UI framework + deployment to Spaces)
• Referenced research papers/credits include:
  • Hinton (knowledge distillation; referenced when discussing DistilBERT)
  • Jonathan Ho et al. (DDPM)
  • Stability AI / SDXL work (stable diffusion XL concepts)
  • Stability AI (stable video diffusion paper)
  • Tongji lab / Alibaba (I2VGen-XL mentioned as open-source codebase)