Video summary

Sesión 10

Main summary

Key takeaways

Educational

Main ideas and concepts (Session 10: Data Analysis)

Purpose of the session

The session introduces general concepts used in data analysis. It covers:

  • Data sets: instances and variables
  • Variable / data types
  • Models: descriptive vs predictive (and common learning tasks)

Forum / interaction and session logistics (before content)

  • A QR code is displayed to join “synchronous sessions.”
  • The forum is intended only for questions specific to the session.
  • No roll call.
  • If questions aren’t answered live, an Infoteca expert will answer them.

Data sets, instances, and variables

  • Data set

    • The raw material of data analysis.
    • Conceptually represented as a table.

  • Instances

    • Correspond to the rows of the table.
    • Represent the objects of interest.
    • Also called: tuples or objects.

  • Variables

    • Correspond to the columns.
    • Also called: attributes or characteristics.
    • Examples (student dataset context):
      • control number, semester, average so far, age, salary, etc.

  • Key relationship

    • You analyze instances using the values of variables (attributes).

Types of variables / data types

1) Categorical (nominal) vs numerical (numeric)

Categorical / nominal

  • Values come from a finite predetermined set.
  • Examples:
    • student ∈ {yes, no}
    • department name ∈ {management, marketing, sales}
    • eye color ∈ {green, brown, blue}
    • gender ∈ {feminine, masculine}
  • Nominal categoricals are used with equality/inequality only (no “greater than” meaning).
  • Note: some identifiers (e.g., a numeric-looking control number) are still categorical if they only identify instances (not for arithmetic).

Numerical

  • Contain integer or real values.
  • Examples:
    • age (integer)
    • salary (real)

2) Categorical subtypes: nominal and ordinal

Nominal categorical

  • Allowed operators: equality and inequality
  • Not meaningful to use: >, < or arithmetic
  • Example comparisons:
    • “coffee == coffee” / “coffee != green”
    • Not meaningful: “coffee > green”

Ordinal categorical

  • Allowed operators: equality/inequality + < and > (there is an order)
  • Arithmetic is still not meaningful (no numeric scale for operations like division)
  • Examples:
    • hardness categories: diamond (very hard), steel (medium), talc (low)
    • academic degree: bachelor < master < doctoral
    • drink size: small < medium < large
  • Limitation:
    • You can say “large > small” but not “medium is twice large.”

3) Numerical subtypes: interval and proportion/ratio

Interval numerical

  • You can use addition/subtraction (differences matter).
  • Multiplication/division not meaningful because there’s no true zero.
  • Examples:
    • calendar dates
    • temperatures in Celsius/Fahrenheit
  • Example:
    • 20°C − 15°C = 5°C (difference makes sense)
  • Not meaningful:
    • “10°C is twice as warm as 5°C” (no true zero)

Proportion / ratio numerical

  • You can use all arithmetic operators, including multiplication/division.
  • Has a true zero.
  • Examples:
    • temperature in Kelvin
    • money/total amounts
    • age, mass, length, electric current
  • Example:
    • A person 100× richer (multiplication/division meaningful)

4) Binary variables (special categorical case)

  • A binary variable is a categorical variable with exactly two values.
  • Subtypes:
    • Symmetric binary: both categories equally important (e.g., gender)
    • Asymmetric binary: one value is “more important” (e.g., smoker increases probability of lung disease → asymmetric)

Models: descriptive vs predictive

What a model represents

Knowledge extracted from data can be:

  • relationships (correlation/similarity/differences)
  • patterns
  • rules

Collectively, this information forms the model.

Two main model types

  • Descriptive models

    • Use unsupervised learning
    • Aim: find structure/patterns in existing data

  • Predictive models

    • Use supervised learning
    • Aim: estimate unknown future/target values

Tasks and methods mentioned

Descriptive analytics tasks

Grouping (clustering)

  • Goal:
    • objects within a group are very similar
    • objects in different groups are very different
  • Note:
    • requires a notion of distance (distance from points to group centers)
  • Example algorithm referenced:
    • k-means / “Caimins”
      • choose number of groups, centroids, measure distances, iterate until groups form
  • Use cases:
    • segment employees to understand behavior and benefits
    • group online bookstore customers for recommendations
    • group news articles by word-frequency patterns

Association rules

  • Goal:
    • discover relationships between categorical attributes
  • Form:
    • “If X takes value D, then Y takes value B”
  • Example domains:
    • shopping basket / products bought together
    • hospital test requests (tests performed together)
  • Measures:
    • Confidence: certainty of the rule
      • intuition: % of times antecedent implies consequent
    • Support: how often the rule occurs in the dataset
      • defined as occurrences where both antecedent and consequent appear, relative to total transactions
  • Also mentioned:
    • sequential association rules (relationships over time, e.g., buy computer → later buy printer)

Correlational analysis

  • Goal:
    • measure similarity/relationship between two numerical variables
  • Metric:
    • correlation coefficient r in [-1, 1]
  • Interpretation:
    • r = 1: perfect positive correlation
    • r < 0: negative correlation
    • r = 0: no correlation

Anomaly detection

  • Goal:
    • identify observations distant from the majority (outliers)
  • Requirements:
    • high detection rate
    • low false alarm rate
  • Example domain:
    • credit card fraud
  • Approach:
    • build a legitimate/fraud profile
    • flag transactions that deviate significantly from expected behavior (e.g., unusual purchase size/frequency)

Predictive analytics tasks

Classification

  • Target (class label) is categorical
  • Goal:
    • predict the class of new/unlabeled instances based on other variables
  • Outputs/model representations mentioned:
    • rules
    • decision tree
    • neural network (as examples of classifier form)
  • Worked concept:
    • training set → learn a classifier
    • classification aims to minimize errors
  • Example applications:
    • bank loan repayment risk (rules derived from training data)
    • iris flower classification
      • interval partitions for petal length/width, rules derived
      • some misclassifications accepted

Regression

  • Target is numerical
  • Goal:
    • learn a real-valued function predicting numeric outcomes
  • Objective:
    • minimize error between predicted and true values
  • Examples:
    • predict travel/warehouse-related numeric outcomes
    • predicting contract costs
  • Error concept:
    • distance from data points to regression line/function

Detailed bullet lists of key “instructional” methodologies

A) Grouping/clustering procedure (k-means-style as described)

  1. Choose the number of groups (k)
  2. Select k centroids (initial center points)
  3. For each data point:
    • compute the distance from the point to each centroid
  4. Assign each point to the nearest centroid
  5. Update centroids based on assignments
  6. Repeat distance + reassignment for multiple iterations
  7. Stop when groups stabilize / centers converge

B) Classification evaluation methodology: cross-validation

  • Split the training set into N folds (example given: 3-fold)
  • For each fold (iteration):
    • use N−1 folds to train
    • use the remaining 1 fold to test
  • Repeat so each fold becomes the test set once
  • Aggregate performance (e.g., average error/metrics across folds)
  • Rationale:
    • reduces risk of misleading results due to one lucky/unlucky split
    • helps prevent overfitting, especially when datasets are small

C) Classification performance measurement: confusion matrix and metrics

  • Use a confusion matrix where:
    • rows/columns reflect real vs predicted classes
    • example classes: (Iris setosa, versicolor, virginica)
  • Count per class:
    • True Positives (TP): predicted class is correct
    • False Positives (FP): predicted class but actually another class
    • False Negatives (FN): actual class but predicted as another class
  • Compute per class:
    • Precision: TP / (TP + FP)
    • Recall / completeness: TP / (TP + FN)
  • Interpretation rule:
    • metrics closer to 1 indicate a better model.

D) “How rules are obtained” from a classifier (decision tree example)

  • Train a decision tree classifier on labeled data.
  • Decision tree training:
    • repeatedly selects the best attribute to split the data
    • creates branches that group instances into purer subsets (often toward single classes)
  • Convert decision tree splits into if-then rules
    • example rule form:
      • If (attribute condition) then (class = outcome)

E) Model selection and reliability (criteria referenced in Q&A)

  • Compare models using metrics, including:
    • accuracy-like measures and recall/completeness
    • mention of F1 score (combined metric)
    • mention of ROC / “area” concept (area under curve using:
      • true positive rate vs false positive rate)
  • Use cross-validation when comparing models
  • Avoid data leakage during preprocessing:
    • apply preprocessing (e.g., cleaning/transforms) based only on training data
    • don’t use information from the test set during transformations

Applications/areas where data analysis can be used

  • Health
  • Commerce
  • Education
  • Biology
  • General statement: any domain where you have data can benefit from analysis to gain knowledge.

Speakers / sources featured

  • Speaker (unnamed): the instructor/presenter of Session 10 (data analysis)
  • Infoteca expert: answers unanswered forum questions (no individual name provided)
  • No other named speakers or organizations are explicitly identified beyond “Infotec.”

Original video