Video summary

27. Learning Vector Quantization | LVQ | LVQ Solved Example - 1 in Soft Computing by Mahesh Huddar

Main summary

Key takeaways

Educational

Main Ideas / Lessons Conveyed

  • Learning Vector Quantization (LVQ) classifies input vectors by assigning each input vector to one of a set of prototype (weight) vectors—interpreted as clusters.
  • The example uses:
    • 5 input vectors
    • 2 classes (class 1 and class 2)
    • Each vector has 4 components, so distance calculations involve four squared terms.
  • The LVQ network is trained by:
    1. Initializing two prototype weight vectors, one per class, using the first two given vectors.
    2. For each remaining input vector:
      • Compute distances to each prototype using Euclidean distance.
      • Select the closest prototype as the winning cluster ((J)).
      • Compare the winning cluster ((J)) with the target class label ((T)).
      • Update the winning prototype (and only that side, consistent with the described formula/sign convention) using the LVQ learning-rate rule.
    3. Repeat epochs until the vectors are classified correctly.

Methodology / Step-by-Step Instructions

1) Setup

  • Given:
    • 5 vectors, each with 4 components: (x_1, x_2, x_3, x_4)
    • 2 classes with target labels:
      • Class 1: (T = 1)
      • Class 2: (T = 2)
  • Learning rate: (\alpha = 0.1)
  • Prototype initialization:
    • Use the first two vectors as initial prototype weights:
      • Class 1 prototype: (W_1)
      • Class 2 prototype: (W_2)
  • Conceptual network mapping:
    • Input layer: (X_1, X_2, X_3, X_4)
    • Cluster comparison outputs: (Y_1) (class 1) and (Y_2) (class 2)

2) For Each Input Vector (Training Iteration)

Let the current input vector be (x), and prototypes be (W_1) and (W_2).

  1. Compute Euclidean distances to each prototype:

    • LVQ distance form: [ D_j = \sum_{i=1}^{4} (w_{ji} - x_i)^2 ]

    • Compute:

      • (D_1): distance to cluster/prototype 1
      • (D_2): distance to cluster/prototype 2

  2. Choose the winner cluster (J):

    • If (D_1 < D_2), then (J = 1)
    • Else if (D_2 < D_1), then (J = 2)

  3. Compare winner (J) with the target (T):

    • Key emphasis:
      • (J) is chosen by distance (the winning cluster).
      • (T) is the true class label.
      • So (J) and (T) are not necessarily equal.
    • If (J = T): classification is correct.
    • If (J \ne T): classification is wrong, and the update uses the alternate sign convention.

  4. Update the prototype using the LVQ learning rule (sign depends on correctness):

    • The update is presented as: [ W^{new} = W^{old} \pm \alpha (x - W^{old}) ]

    • Sign convention described:

      • If the input is mapped incorrectly ((J \ne T)): use “minus”
      • If the input is mapped correctly ((J = T)): use “plus”
    • Apply the update to the relevant prototype components for (i = 1..4).

3) Epoch Repetition (Training Loop)

  • Continue with the remaining input vectors using the same process:
    • “Continue with the next input vector…”
  • Repeat training epoch by epoch:
    • “Repeat again and again until all these input vectors were classified correctly.”
  • Final output:
    • The converged weights are the final trained prototypes.

What Happens in the Worked Example (High Level)

  • Input vector 1:
    • Compute (D_1) and (D_2)
    • Winner chosen by smaller distance, but winner didn’t match the target
    • Therefore, weights were updated using the incorrect-case sign
  • Input vector 2:
    • Distances computed; winner chosen
    • Mapped incorrectly, so weights updated accordingly
  • Input vector 3:
    • Distances computed; winner chosen
    • Winner matched the target ((J = T))
    • Weights updated using the correct-case sign
  • The video notes that after “only first epoch,” not all classifications are correct yet—so training must continue.

Speakers / Sources

  • Mahesh Huddar (video creator; referenced in the title)
  • The video narrator (not separately identified; speaking voice aligns with Mahesh Huddar)

Original video