Lecture 5 - LIME from Explainable AI (XAI) explained

Summary

The lecture introduces LIME, a model-agnostic method for explaining why a machine-learning model made a particular prediction. Rather than explaining a model’s overall behavior, LIME focuses on one prediction at a time and builds a simple, human-readable approximation of the model near that example.

Main Ideas

• Local rather than global explanations: A model may behave differently in different regions of its feature space. Overall trends do not necessarily explain why a specific patient, sentence, or image received a particular prediction.
• Interpretable explanations: The explanation should identify features a person can understand—for example, BMI or glucose for a patient, words in a sentence, or image regions.
• Model-agnostic approach: LIME can be applied to many kinds of predictive models; it does not require a particular model architecture.
• Surrogate-model intuition: Complex models are often difficult to interpret directly. LIME approximates the model locally with a simpler, interpretable model, commonly a linear model. The surrogate’s feature weights indicate which features mattered most around the example being explained.

LIME’s General Workflow

1. Choose an instance to explain. This could be a row in a patient dataset, a text passage, or an image.
2. Create perturbed samples around it. LIME modifies or masks parts of the instance to produce nearby examples.
3. Query the original model. The model predicts an outcome for each perturbed sample.
4. Weight samples by proximity. Nearby samples receive more influence than samples farther from the instance.
5. Fit a simple local surrogate. LIME finds an interpretable model that approximates the original model’s predictions in the instance’s neighborhood.
6. Read the explanation. The surrogate’s feature weights indicate which features supported or opposed the prediction for that instance.

The lecture describes the mathematical objective as balancing two aims: local fidelity (how well the surrogate matches the original model near the instance) and simplicity (keeping the surrogate interpretable). In notation, this is presented as minimizing a weighted loss, (L(f, g, \pi_x)), plus a complexity penalty, (\Omega(g)). The proximity weighting, (\pi_x), reduces the influence of samples distant from the example being explained.

How Perturbations Work for Different Data Types

• Tabular data: Change feature values to generate nearby synthetic records, such as altered glucose, BMI, or age values. The lecture’s diabetes example shows how a local explanation might indicate that age and BMI pushed one prediction toward diabetes, while other feature values pushed it away. These are example-specific contributions, not universal medical rules.
• Text: Split text into words or tokens, then create perturbed versions by removing words. The model’s responses to these versions help identify which words influenced the classification.
• Images: Divide an image into superpixels—interpretable regions—and create variations by switching regions on or off, often by graying out some regions. The explanation highlights regions that most influenced the model’s prediction.

Examples and Lessons

• Patient classification: Two patients may receive different predictions even when they share some characteristics. A local explanation can show which feature distinguishes a particular case, such as body weight in one region or exercise duration in another.
• Urgent text classification: In a hypothetical system prioritizing requests, words in a message about a medical emergency may help explain why it is ranked as more important than a routine service inquiry.
• Husky classified as a wolf: The lecture describes an example where LIME highlights snow rather than the husky’s defining features. This suggests the model may have learned to associate snow with wolves, illustrating how high accuracy alone does not guarantee trustworthy behavior.
• Image classification: Examples involving guitars, a dog, a frog, billiard balls, and a balloon show how highlighted image regions can help reveal what visual evidence the model is using.

Limitations Discussed

• Local explanations do not describe the whole model. What matters near one example may not represent model behavior elsewhere.
• Defining the neighborhood is difficult. The chosen locality or distance scale affects which samples influence the explanation.
• The linear approximation may be inaccurate. A model can remain nonlinear even in a small neighborhood, making a simple surrogate a poor fit.
• Explanations can be unstable. Because perturbations are sampled, different samples may produce different explanations for the same instance.
• Perturbations can be unrealistic or biased. Synthetic examples may violate real-world constraints—for instance, by generating an implausible blood-pressure value—and can therefore produce misleading explanations.

The lecture emphasizes that LIME is useful for inspecting black-box predictions, but its explanations should be understood as local approximations, not complete accounts of a model’s behavior.

Speakers and Sources Featured

• Speaker: One unnamed lecturer or narrator. No other speaking voices are identified.
• Sources and models mentioned:
  • The original LIME paper, “Why Should I Trust You? Explaining the Predictions of Any Classifier,” described in the lecture as published on arXiv.
  • Google’s Inception neural network, used in an image-classification example.
  • The LIME GitHub repository and two blog articles, referenced as further reading. The subtitles do not provide their titles.