Week 15 Info: Pretrained Models
Congratulations, you have made it to the last week of the class! During this week will discuss how to work with the latest and most advanced models that are based on the transformer architecture. These models are extraordinarily expensive to train, requiring huge compute resources and copious quantities of data. However, once you have the model, it is possible to adjust it to your specific tasks through a procedure called fine-tuning. We will learn about the transformer architecture, which is what made these models possible, introduce several model classes which use transformers, and introduce methods for fine-tuning these models.
You can read more about the plan for the week and the reading/resources in Module 15.
Lab 8 is now optional. If you elect to complete it, it will replace the lowest grade on labs 1-7. It is available here: