EDA using Bivariate and Multivariate Analysis | Day 21 | 100 Days of Machine Learning

Main ideas / lessons conveyed

• The video explains Exploratory Data Analysis (EDA) using bivariate and multivariate techniques, focusing on how to choose the right visualization depending on data types.
• It emphasizes that no single technique fits every dataset, demonstrated by switching between several built-in datasets.
• The core EDA workflow shown:
  • Identify variable types (numeric vs categorical).
  • Form questions about relationships (e.g., correlation/trends, group comparisons, survival rates).
  • Use appropriate plots to answer those questions.
• It demonstrates practical insights from the Titanic dataset, including survival differences by:
  • Class
  • Gender
  • Children
  • Age
• Plot techniques used on Titanic (and beyond) include:
  • Scatter plots with coloring for multivariate relationships
  • Bar/box plots for grouped comparisons
  • PDF/density plots to compare distributions
  • Contingency tables + heatmaps for cross-tabs
  • Derived metrics (e.g., percentages via groupby)
  • Pair plots for quick all-numeric relationship exploration
  • Line plots for time-series data
  • Pivot-table-style visualizations (heatmap/clustermap style) for time-based aggregates

---

Methods and instructions

1) Choose plots based on variable types (conceptual rule)

• If you have two numeric columns:
  • Use a scatter plot to study relationship (trend/linear correlation).
• If you have numeric + categorical:
  • Use a bar plot (or similar aggregation plot) to compare numeric values across categories.
  • Optionally refine with a box plot to show distribution spread per category.
• If you have two categorical variables:
  • Use a heatmap / cross-tab (contingency) to see frequency or rate patterns.
• If you have time/date as the x-axis:
  • Use a line plot to show movement over time.
• If you have many numeric variables:
  • Use a pair plot to generate scatter plots for all pairwise combinations automatically.

---

2) Bivariate numeric vs numeric: Scatter plot relationship

• Use a scatter plot to test relationships between two numerical features.
• Example shown (Tips dataset):
  • Plot total bill (x) vs tip (y).
  • Observe an overall positive trend (larger bills generally mean larger tips).
  • Look for exceptions/outliers.

---

3) Multivariate numeric relationships using scatter color/size/style

• Extend scatter plots into multivariate analysis by mapping additional columns to visual properties:
  • Color → categorical feature (e.g., gender)
  • Size → another attribute (quantitative or categorical)
  • Style/marker → another categorical feature (e.g., smoker vs non-smoker)
• Example (Tips dataset):
  • Scatter: total bill vs tip
  • Color by gender
  • Marker style by smoking status
  • Size by party size
• Interpret patterns:
  • Color suggests which groups have higher/lower tips
  • Marker styles differ for smokers vs non-smokers
  • Larger circles indicate larger parties

---

4) Bar plots for numeric value aggregated by categorical groups (Titanic example)

• Goal: compare an average metric across classes.
• Conceptual setup:
  • x-axis: categorical variable (e.g., Pclass)
  • y-axis: numeric variable (e.g., Age)
  • aggregate using mean (average age per class)
• Multivariate extension:
  • Add another categorical grouping (e.g., gender) and compute separate averages per class.

---

5) Box plots for distribution comparison (multivariate extension)

• Use box plots to compare how a numeric variable is distributed across categorical groups.
• Example:
  • Compare distribution spread (median, quartiles, outliers) for categories like gender or class.

---

6) Probability density function (PDF) to compare distributions (survival example)

• Use a density plot to compare distributions of a variable across two groups:
  • Example: survivors vs non-survivors (e.g., age distribution)
• If the group label doesn’t exist:
  • Create a new feature / filtered dataframe, such as survival group derived from Titanic’s survived/died condition.
• Plot:
  • Two density curves (e.g., blue vs orange)
• Interpret:
  • Compare where each group’s probability mass lies
  • Highlight patterns such as children/younger ages having different survival vs death probabilities

---

7) Categorical-categorical analysis: contingency table + heatmap

• Build a cross-tabulation between two categorical variables:
  • Example: Pclass (rows) vs Survived (columns)
• Visualize with a heatmap:
  • Color intensity reflects how common each combination is.
• Explain a limitation:
  • Raw counts can mislead if group sizes differ.
• Fix: convert counts to percentages using group-wise computations:
  • survival rate = (count in group) / (total in class) × 100
• Then plot/interpret the adjusted rates.

---

8) Additional derived comparisons (grouped rates/percentages)

• Demonstrated examples:
  • Survival rate by sex (female vs male)
  • Survival patterns involving children/age categories
  • Compare across ports/embarkation points (e.g., Southampton vs others) using rates
• Common instruction pattern:
  • Define/filter groups (children vs adults, male vs female, etc.)
  • Compute group-wise survival percentages
  • Visualize and compare results

---

9) Relationship among multiple categorical/continuous variables

a) Scatter + hue/size/style

• Use the same multivariate scatter mapping strategy described earlier.

b) “Similarity” via heatmap-like structure

• Demonstrate correlation/association-style visualization:
  • Compute relationships between binned/feature-defined groups
  • Use a heatmap to interpret which combinations behave similarly

---

10) Pair plot for many numeric variables (automatic pairwise exploration)

• If there are multiple numeric columns:
  • Use a pair plot to automatically generate scatter plots for every numeric pair.
• Key features:
  • Includes diagonal plots (often histograms/distributions for each variable)
  • May optionally color points by a categorical label (multivariate extension)

---

11) Line plots for time-series data (Flights example)

• For time-based data:
  • Group by year (or year-month) and compute totals/aggregates
• Use a line plot:
  • x-axis: time (year/month)
  • y-axis: numeric measure (e.g., passenger count)
• Interpret:
  • Growth trends over years
  • Seasonality peaks (e.g., higher passenger counts in summer months)

---

12) Heatmaps / cluster maps for aggregated time patterns

• Approach:
  • Transform time-series into a matrix using pivot-like transformations (months vs years)
• Visualize:
  • Heatmap to view intensity by month/year
  • Cluster map to group similar rows/columns
• Interpret:
  • Identify which months behave similarly (e.g., January/February-like)
  • Observe which years have comparable patterns

---

Speakers or sources featured (end)

• Speaker/Host: Unnamed instructor/creator (“Hello guys…”) conducting the tutorial
• Primary visualization tools referenced: Python visualization libraries (e.g., Matplotlib/Seaborn-style tools; exact names not clearly specified)
• Datasets used within the tutorial:
  • Tips
  • Flights
  • Titanic
  • Iris-like flower dataset (referred to as “agricultural business data / petal & sepal / features”)