YoVDO

Knowledge Distillation Demystified: Techniques and Applications

Offered By: Snorkel AI via YouTube

Tags

Machine Learning Courses Transfer Learning Courses Model Optimization Courses Model Compression Courses Synthetic Data Courses Data-Centric AI Courses Snorkel AI Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore the powerful technique of knowledge distillation for optimizing machine learning models, particularly in natural language processing (NLP), in this 24-minute video presentation by Charlie Dickens, an applied research scientist at Snorkel AI. Learn about the fundamental concepts, benefits, methodologies, and real-world applications of knowledge distillation, which transfers knowledge from a large, complex model (the teacher) to a smaller, more efficient model (the student). Understand the two main steps of knowledge distillation: extraction and transfer. Discover how to identify target skills and curate seed knowledge for effective student model training. Examine various techniques for knowledge extraction, including teacher labeling, hidden representations, synthetic data, and feedback. Gain insight into the latest research and advancements in knowledge distillation, with a focus on the innovative data-centric approach being developed at Snorkel AI.

Syllabus

Knowledge Distillation Demystified: Techniques and Applications


Taught by

Snorkel AI

Related Courses

Solving the Last Mile Problem of Foundation Models with Data-Centric AI
MLOps.community via YouTube
Foundational Models in Enterprise AI - Challenges and Opportunities
MLOps.community via YouTube
Model Distillation - From Large Models to Efficient Enterprise Solutions
Snorkel AI via YouTube
Curate Training Data via Labeling Functions - 10 to 100x Faster
Snorkel AI via YouTube
Task Me Anything: Revolutionizing Multimodal Model Benchmarking
Snorkel AI via YouTube