Knowledge Distillation Demystified: Techniques and Applications
Offered By: Snorkel AI via YouTube
Course Description
Overview
Explore the powerful technique of knowledge distillation for optimizing machine learning models, particularly in natural language processing (NLP), in this 24-minute video presentation by Charlie Dickens, an applied research scientist at Snorkel AI. Learn about the fundamental concepts, benefits, methodologies, and real-world applications of knowledge distillation, which transfers knowledge from a large, complex model (the teacher) to a smaller, more efficient model (the student). Understand the two main steps of knowledge distillation: extraction and transfer. Discover how to identify target skills and curate seed knowledge for effective student model training. Examine various techniques for knowledge extraction, including teacher labeling, hidden representations, synthetic data, and feedback. Gain insight into the latest research and advancements in knowledge distillation, with a focus on the innovative data-centric approach being developed at Snorkel AI.
Syllabus
Knowledge Distillation Demystified: Techniques and Applications
Taught by
Snorkel AI
Related Courses
Solving the Last Mile Problem of Foundation Models with Data-Centric AIMLOps.community via YouTube Foundational Models in Enterprise AI - Challenges and Opportunities
MLOps.community via YouTube Model Distillation - From Large Models to Efficient Enterprise Solutions
Snorkel AI via YouTube Curate Training Data via Labeling Functions - 10 to 100x Faster
Snorkel AI via YouTube Task Me Anything: Revolutionizing Multimodal Model Benchmarking
Snorkel AI via YouTube