Week 6 Info: Resampling Methods
Welcome to Week 6! During this week, we will learn about resampling methods, tools which have been instrumental in the modernization of statistics and in its interface with machine learning in the 21st century. These tools are used for several distinct purposes, predicting out of sample performance in the context of model selection, hyperparameter optimization (will be a bigger subject next week, but it is very similar to model selection), and uncertainty quantification. The basics of resampling are simple, but there are lots of subtleties, pitfalls (there is a lot of potential for introducing bias and unexpected data leakage), and advanced techniques to mitigate these pitfalls. We will start from the basics and build up from there. There will be a coding vignette (and possibly some bonus slides on the advanced cross-validation methods if we don’t get there
You can read more about the plan for the week and the reading/resources in Module 6.
A note on project proposals: I had to make some adjustments to the class databricks environment due to uncertainty about our credit budget. This should be resolved shortly. If you need an extra week for your proposals, you may have it.
Lab 4 is due in two weeks, Sunday at midnight.