YoVDO

Unlocking Mixture of Experts - From One Know-it-all to a Group of Jedi Masters

Offered By: EuroPython Conference via YouTube

Tags

Mixture-of-Experts Courses Machine Learning Courses Predictive Modeling Courses Divide-and-Conquer Courses Model Optimization Courses Ensemble Models Courses Pre-trained Models Courses Fine-Tuning Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Embark on an exhilarating journey exploring the Mixture of Experts (MoE) technique in this 31-minute conference talk at EuroPython 2024. Delve into the practical and intuitive next step for elevating predictive powers of generalized know-it-all models, particularly in critical domains like healthcare. Discover the powerful Divide and Conquer principle behind MoE, its limitations, pros, and cons. Progress through a captivating exploration of insights, intuitive reasoning, and solid mathematical underpinnings, enriched with interesting examples. Survey the landscape from ensemble models to stacked estimators, gradually ascending to MoE. Explore challenges, alternative routes, and learn when to apply MoE effectively. Conclude with a business-oriented discussion on metrics around cost, latency, and throughput for MoE models. Gain access to resources for diving into pre-trained MoE models, fine-tuning them, or creating your own from scratch.

Syllabus

Unlocking Mixture of Experts : From 1 Know-it-all to group of Jedi Masters — Pranjal Biyani


Taught by

EuroPython Conference

Related Courses

GShard- Scaling Giant Models with Conditional Computation and Automatic Sharding
Yannic Kilcher via YouTube
Learning Mixtures of Linear Regressions in Subexponential Time via Fourier Moments
Association for Computing Machinery (ACM) via YouTube
Modules and Architectures
Alfredo Canziani via YouTube
Stanford Seminar - Mixture of Experts Paradigm and the Switch Transformer
Stanford University via YouTube
Decoding Mistral AI's Large Language Models - Building Blocks and Training Strategies
Databricks via YouTube