LoRA Fine-tuning Explained - Choosing Parameters and Optimizations
Offered By: Trelis Research via YouTube
Course Description
Overview
Dive into a comprehensive video tutorial on LoRA fine-tuning for machine learning models. Explore recent developments in Mistral v0.3 and Phi-3 models before delving into full fine-tuning techniques. Learn the intricacies of LoRA, including how to select optimal alpha and rank parameters. Discover strategies for choosing fine-tuning parameters such as learning rate, schedule, and batch size. Gain insights into advanced optimizations like rank stabilized LoRA, loftQ, and LoRA+. Follow along with a practical demonstration using SFTTrainer from TRL to run training sessions. Access additional resources and support, as well as a companion notebook, to enhance your understanding of LoRA fine-tuning techniques.
Syllabus
Welcome
Mistral v0.3
Phi-3 models
Full fine-tuning
LoRA
Picking LoRA alpha and rank
Running training with SFTTrainer from TRL
Taught by
Trelis Research
Related Courses
Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and OptimizationDeepLearning.AI via Coursera Machine Learning in the Enterprise
Google Cloud via Coursera Art and Science of Machine Learning 日本語版
Google Cloud via Coursera Art and Science of Machine Learning auf Deutsch
Google Cloud via Coursera Art and Science of Machine Learning en Español
Google Cloud via Coursera