LECTURE 26

Hyperspectral Image Classification

The lecture introduces hyperspectral image classification, explains why hyperspectral data requires specialized methods, and compares traditional classifiers with CNN-based approaches.

What Makes Hyperspectral Images Different

• RGB images contain three bands, while multispectral images contain a limited number of bands.
• Hyperspectral images capture hundreds of narrow, often contiguous spectral bands. Each pixel therefore has a detailed spectral signature.
• Hyperspectral data can be viewed as a 3D cube: two dimensions represent spatial location, and the third represents spectral information.
• Because different materials reflect light differently, their spectral signatures can act like fingerprints and support material identification. Higher spectral resolution can help distinguish materials that appear similar in RGB or multispectral images.

Applications include:

• Fine-grained crop monitoring and classification
• Mineral study, exploration, and monitoring of mining activity
• Military surveillance and tracking of materials of interest

Challenges and Dimensionality Reduction

Hyperspectral data also presents several difficulties:

• Adjacent bands are often highly correlated, creating redundant information and increasing computational cost.
• High-dimensional inputs generally require more training examples, but labeled data for fine-grained material or crop classification is often limited.
• This combination can increase the risk of overfitting.

Dimensionality reduction can help reduce redundancy and computation, and potentially limit overfitting. Two broad approaches are:

• Feature selection: Choose a subset of the original bands.
• Feature extraction: Transform the original bands into a smaller set of new features, such as through PCA.

The lecturer cautions against discarding so much information that the benefit of using hyperspectral data is lost. Dimensionality reduction may be an important preprocessing step before classification.

Meaning of “Classification”

In this context, hyperspectral image classification usually means pixel-wise classification: assigning a class to each pixel. This is similar to what is commonly called semantic segmentation.

Traditional, Pre-Deep-Learning Methods

• K-nearest neighbors (KNN): Assigns a class based on nearby labeled samples. It is simple, but sensitive to high dimensionality.
• Support vector machines (SVMs): Widely used and can work effectively with limited labeled samples, though performance may be limited on complex scenes.
• Decision trees and random forests: Partition the feature space. Random forests improve the robustness of individual trees.
• Spectral Angle Mapper (SAM): Compares the angle between a reference spectrum and a target spectrum. Related to cosine similarity, it is comparatively illumination-invariant and useful for material identification, though complex scenes can still be challenging.

CNN Approaches for Hyperspectral Data

CNN architectures must account for both the spatial and spectral structure of hyperspectral images.

1D CNN

• Operates along the spectral dimension and learns spectral patterns.
• Is computationally efficient but ignores spatial neighborhoods.
• May be suitable when spatial resolution is coarse or spatial context is not especially useful.

2D CNN

• Uses spatial patches to learn texture and contextual patterns.
• A filter spans all input channels. For example, with an input of (H \times W \times 200), a (3 \times 3) filter spans all 200 channels.
• Works well for images with relatively few spectral channels, but does not explicitly model spectral relationships in the same way as a 3D CNN.

Channel Attention

Channel attention adaptively gives more weight to informative channels and less weight to less useful ones. The process is:

1. Start with a feature tensor of shape (H \times W \times C).
2. Use global average pooling to collapse the spatial dimensions, producing a channel-wise descriptor of shape (1 \times 1 \times C).
3. Pass the descriptor through a lightweight learnable network to estimate the importance of each channel.
4. Multiply each channel in the original feature tensor by its learned weight, producing a recalibrated (H \times W \times C) tensor.

3D CNN

• Convolves across spatial and spectral dimensions jointly, capturing their interactions.
• Its output retains a spectral-depth dimension.
• Is widely used for hyperspectral classification, although pure 3D CNNs can be computationally expensive and may produce large feature representations.

Hybrid 2D–3D CNNs

Hybrid architectures aim to combine the strengths of 3D and 2D convolutions:

• A 3D branch extracts joint spatial–spectral features.
• A 2D branch learns higher-level spatial structures.
• A fusion step concatenates or adds features from the branches before passing them to classification layers.

The lecture illustrates this with an input of (25 \times 25 \times 30):

1. A 3D convolution using six (7 \times 7 \times 7) filters produces a feature tensor of (19 \times 19 \times 24 \times 6).
2. Reshaping the last two dimensions gives (19 \times 19 \times 144).
3. A 2D convolution with (3 \times 3) filters then produces (17 \times 17 \times 80).
4. In another branch, successive 3D convolutions are applied before reshaping and applying a 2D convolution.
5. The branch outputs are fused and passed to classification layers.

The lecture concludes that the choice of architecture depends on whether spectral information, spatial context, or both are most important, as well as on the computational cost of the model.

Speakers and Sources

• Speaker: Unnamed lecturer presenting NPTEL Lecture 26 on hyperspectral image classification
• Source: NPTEL, IIT Delhi