Neural Networks Based on the Human Brain

Summary

1. Biological nervous system as a model for neural networks

The lecture introduces artificial neural networks (ANNs) by describing how biological neurons receive, process, and transmit signals.

• Sensory organs, such as the eyes, ears, and skin, provide inputs.
• Dendrites receive signals, the soma (cell body) processes them, and the axon carries the signal onward to another neuron or an organ.
• Signals pass between neurons at a synapse, the junction between an axon terminal and the next neuron’s dendrites.
• A neuron fires only when its processed signal reaches a threshold. Once fired, the signal travels along the axon.
• Biological feedback loops can send output from a motor organ back into the system, either directly to central processing or through a sensory organ.

These biological parts serve as an analogy for ANN components: dendrites correspond to inputs, the soma to processing, and the axon to output.

2. Synaptic transmission

The lecture distinguishes between two ways signals can cross a synapse:

• Chemical transmission: Described as the dominant form in humans and other mammals. It uses neurotransmitters and is slower, which the lecturer associates with learning and more complex processing.
• Electrical transmission: Described as faster and potentially bidirectional, and associated with rapid reflexes.

The lecturer presents these as a speed-versus-function contrast: chemical transmission is slower but suited to more involved processing, while electrical transmission supports rapid responses.

3. How an artificial neuron works

An artificial neuron combines its inputs mathematically:

1. Multiply each input (x_i) by its corresponding weight (w_i).
2. Add the weighted inputs together.
3. Add a bias.
4. Pass the result through an activation function.
5. Send the activated value to the next layer or the output.

During training, the model updates its weights so its predictions better match known target values. The lecture emphasizes that the model learns an input-to-output mapping. Its predictions depend on how well it was trained and on choices such as the activation function and network structure.

4. Activation functions

The lecture introduces four activation functions:

• Linear: (f(x)=x); the output follows the input.
• Sigmoid: Produces values between 0 and 1.
• Tanh (hyperbolic tangent): Produces values between −1 and 1.
• ReLU (rectified linear unit): Produces zero for negative inputs and positive values for positive inputs.

The presenter identifies ReLU and tanh as commonly used choices and notes that selecting an activation function is part of hyperparameter tuning.

5. Network layers and architectures

A basic ANN has three parts:

• Input layer: Receives the features; the lecture treats it mainly as a pass-through stage.
• Hidden layer(s): Process the data using weighted sums, biases, and activation functions.
• Output layer: Produces a prediction or classification.

The presenter describes three network styles:

• Wide neural network: Has a few hidden layers, each with many neurons. The lecture suggests this can suit relatively less complex problems.
• Deep neural network: Has many hidden layers, generally with fewer neurons per layer than in the wide-network example. It is presented as useful for more complex tasks, such as image processing and natural-language processing.
• Narrow neural network: Has few hidden layers and few neurons.

The number of inputs can also be expanded by supplying transformed features—for example, squared inputs, products of inputs, or trigonometric functions—instead of relying on biological-style branching dendrites.

6. Classification and regression

The lecture identifies two common ANN tasks:

• Classification: Assigns data to categories, such as spam/not spam or whether a temperature exceeds a threshold.
• Regression: Predicts a continuous value, such as a future price or a continuously varying control response.

Illustrative classification demonstrations compare activation functions. The presenter reports that the linear model performs poorly in the example, while sigmoid, tanh, and ReLU provide better separation, with tanh and ReLU performing particularly well. A separate regression demonstration shows a deeper network producing predictions that closely match the target values.

7. Battery-health case study and model-development workflow

The main applied example concerns predicting battery state and lifetime. The lecture explains why battery age alone is insufficient: two batteries of the same calendar age may have undergone very different numbers of charge–discharge cycles and therefore have different health.

The described workflow is:

1. Choose the inputs: Use voltage and current as initial measurements.
2. Perform feature selection: Keep variables that significantly affect the target and remove less useful ones. The lecture’s water-heating analogy illustrates that collecting every possible measurement can add cost and create reliability problems without improving the model.
3. Reduce dimensionality with PCA: Use principal component analysis to reduce redundant or unnecessary data while retaining useful information.
4. Normalize the data: Adjust the data’s numerical ranges.
5. Process the data through the network: Hidden layers calculate weighted sums, add biases, and apply activation functions.
6. Use a cascaded prediction structure: A sequence of models predicts intermediate quantities—such as state of charge, power density, and energy density—then uses those results along with the original inputs to estimate battery life.
7. Train and evaluate the model: Compare predicted values with actual values and adjust the model design as needed.

The lecturer shows examples comparing narrow and wide networks, along with other machine-learning approaches such as support vector machines, decision trees, and Gaussian process regression. In the examples presented, the wide neural network generally gives a closer fit than the narrow network.

8. Measuring performance and tuning hyperparameters

The lecture recommends several numerical metrics for comparing models:

• MAE (mean absolute error): The average absolute difference between actual and predicted values.
• MSE (mean squared error): The average squared difference.
• RMSE (root mean squared error): The square root of MSE. Lower is better, with zero indicating a perfect fit.
• (R^2): A measure of fit; the lecture describes a value of 1 as ideal.

The presenter distinguishes weights, which are learned during training, from hyperparameters, which are set before training. Examples of hyperparameters include the number of hidden layers and neurons, the learning rate, the selected features, and the activation function.

Hyperparameter tuning involves trying different settings and selecting those that reduce the error. The lecture names Adam as an optimization method.

Speakers and sources featured

• Dr. Pritamik (name as transcribed in the subtitles), from the School of Electrical Engineering, is the lecturer and presenter.
• The lecturer cites the research paper “Capacity degradation influence state of charge and life cycle prediction through tri-layered wide neural network model with extensive energy storage characterization,” published in the Journal of Energy Storage. The lecturer says it was published with PhD scholar Sarinia S. T. under the lecturer’s supervision.