Summary of Generalized Linear Models Tutorial 2 Video 2
Overfitting can occur in linear regression and logistic regression models when there is less data and more features.
- Regularization is used to shrink parameter theta towards zero to reduce overfitting.
- L2 Regularization (Ridge Regularization) penalizes squared parameters to suppress large theta values.
- L1 Regularization (Lasso Regularization) penalizes the absolute value of parameters to encourage sparse solutions.
- Cross-validation is used to select the optimal hyperparameter beta for Regularization.
- L2 penalty does not prefer one feature over another, while L1 penalty achieves feature selection by throwing away some features.
- The optimal beta value is selected based on accuracy performance on validation data.
- logistic regression models with L1 and L2 penalties are evaluated and compared in terms of feature selection and accuracy performance.
Notable Quotes
— 01:30 — « To overcome overfitting issue, people use regularization. »
— 01:42 — « It shrinks the parameter theta in GLMs towards zero. Thus, it can reduce overfitting. »
— 02:39 — « So one intuitive way of improving the parameter estimate is to suppress these large theta values. »
— 02:55 — « Formally written as this. We add a penalty term to the log-likelihood, maximizing the log likelihood, L prime, will lead to minimizing the magnitude of theta i. »
— 05:02 — « Another popular regularization is L1 regularization, which is also known as Lasso regularization. »
Category
Educational