Ridge Regression and the Lasso with the Hitters Data

Click here for a series of short videos on ridge regression and the lasso in ‘sklearn’, covering regularization paths, cross-validated selection of the penalty, and ‘RidgeCV’/‘LassoCV’

I have recorded a series of coding videos on regularized regression (these are technically from a past semester, so keep that in mind if I say anything not current). They follow a modified version of the chapter 6 coding notebook in ‘ISLP’. I have cut the parts on forward, backward, and subset selection and on dimensionality reduction, and I have added more on ‘RidgeCV’ and ‘LassoCV’, which you will want for your homework. The dataset is the ‘Hitters’ dataset, which we introduced back in the ‘statsmodels’ vignette in week 3. We go through first how to use general cross-validation to select the best value of the regularization parameter, then we show how to estimate the out of sample accuracy and the uncertainty in it, and we also look at nested cross-validation.

There will be an additional Lasso Vignette (exceeded my Youtube upload limit for the day) shortly.

Video Link	Runtime
Lasso Vignette 3: CV Error vs Alpha with ‘mse_path_’	4:33
Lasso Vignette 2: The Lasso Solution Path and Sparsity	5:33
Lasso Vignette 1: ‘LassoCV’ and the Cross-Validated Alpha	5:16
Ridge Vignette 8: Error for a Tuned Model and Nested CV	6:23
Ridge Vignette 7: Fast Leave-One-Out CV with ‘RidgeCV’	5:16
Ridge Vignette 6: Scoring Metrics, R-squared vs MSE	3:14
Ridge Vignette 5: K-Fold CV and Standard Errors	5:21
Ridge Vignette 4: Validation Sets and ‘GridSearchCV’	6:45
Ridge Vignette 3: Plotting and Labeling the Path with ‘adjustText’	8:58
Ridge Vignette 2: The Ridge Regularization Path	6:50
Ridge Vignette 1: Setup and Hitters Data Prep	7:01

The code for these can be found in the following jupyter notebooks: regularization-hitters.ipynb or regularization-hitters-live.ipynb