YoVDO

LoRA Fine-tuning Explained - Choosing Parameters and Optimizations

Offered By: Trelis Research via YouTube

Tags

LoRA (Low-Rank Adaptation) Courses Machine Learning Courses Hyperparameter Tuning Courses Fine-Tuning Courses Phi-3 Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Dive into a comprehensive video tutorial on LoRA fine-tuning for machine learning models. Explore recent developments in Mistral v0.3 and Phi-3 models before delving into full fine-tuning techniques. Learn the intricacies of LoRA, including how to select optimal alpha and rank parameters. Discover strategies for choosing fine-tuning parameters such as learning rate, schedule, and batch size. Gain insights into advanced optimizations like rank stabilized LoRA, loftQ, and LoRA+. Follow along with a practical demonstration using SFTTrainer from TRL to run training sessions. Access additional resources and support, as well as a companion notebook, to enhance your understanding of LoRA fine-tuning techniques.

Syllabus

Welcome
Mistral v0.3
Phi-3 models
Full fine-tuning
LoRA
Picking LoRA alpha and rank
Running training with SFTTrainer from TRL


Taught by

Trelis Research

Related Courses

How to Do Stable Diffusion LORA Training by Using Web UI on Different Models
Software Engineering Courses - SE Courses via YouTube
MicroPython & WiFi
Kevin McAleer via YouTube
Building a Wireless Community Sensor Network with LoRa
Hackaday via YouTube
ComfyUI - Node Based Stable Diffusion UI
Olivio Sarikas via YouTube
AI Masterclass for Everyone - Stable Diffusion, ControlNet, Depth Map, LORA, and VR
Hugh Hou via YouTube