LoRA Fine-tuning Explained - Choosing Parameters and Optimizations
Offered By: Trelis Research via YouTube
Course Description
Overview
Dive into a comprehensive video tutorial on LoRA fine-tuning for machine learning models. Explore recent developments in Mistral v0.3 and Phi-3 models before delving into full fine-tuning techniques. Learn the intricacies of LoRA, including how to select optimal alpha and rank parameters. Discover strategies for choosing fine-tuning parameters such as learning rate, schedule, and batch size. Gain insights into advanced optimizations like rank stabilized LoRA, loftQ, and LoRA+. Follow along with a practical demonstration using SFTTrainer from TRL to run training sessions. Access additional resources and support, as well as a companion notebook, to enhance your understanding of LoRA fine-tuning techniques.
Syllabus
Welcome
Mistral v0.3
Phi-3 models
Full fine-tuning
LoRA
Picking LoRA alpha and rank
Running training with SFTTrainer from TRL
Taught by
Trelis Research
Related Courses
How to Do Stable Diffusion LORA Training by Using Web UI on Different ModelsSoftware Engineering Courses - SE Courses via YouTube MicroPython & WiFi
Kevin McAleer via YouTube Building a Wireless Community Sensor Network with LoRa
Hackaday via YouTube ComfyUI - Node Based Stable Diffusion UI
Olivio Sarikas via YouTube AI Masterclass for Everyone - Stable Diffusion, ControlNet, Depth Map, LORA, and VR
Hugh Hou via YouTube