CMU Advanced NLP: How to Use Pre-Trained Models

Explore the intricacies of utilizing pre-trained models in this guest lecture by Aditi Raghunathan for CMU's Advanced NLP course. Delve into the reasons behind pretraining, examine satellite remote sensing applications, and compare different fine-tuning methods. Investigate empirical observations, linear probing techniques, and various regularizers. Engage with in-context learning concepts, develop mental models, and understand latent concepts and problem distributions. Participate in a thought-provoking discussion and question session to deepen your understanding of advanced natural language processing techniques.

Intro
Why pretraining
Satellite Remote Sensing
Fine Tuning Pretrained Models
Comparing Two Methods of Fine Tuning
Empirical Observations
Linear probing
Finetuning
Regularizers
Linear Probe
Questions
Discussion
In Context Learning
Whats Happening Here
Mental Model
Latent Concept
Problem Distribution