YoVDO

Federated Hyperparameter Tuning - Challenges, Baselines, and Connections

Offered By: Stanford University via YouTube

Tags

Federated Learning Courses Machine Learning Courses Hyperparameter Optimization Courses Hyperparameter Tuning Courses Bayesian Optimization Courses Neural Architecture Search Courses

Course Description

Overview

Explore the challenges, baselines, and connections to weight-sharing in federated hyperparameter tuning through this comprehensive conference talk. Delve into the complexities of tuning hyperparameters in federated learning environments, where models are trained across distributed networks of heterogeneous devices. Learn about key challenges in federated hyperparameter optimization and discover how standard approaches can be adapted to form baselines. Gain insights into a novel method called FedEx, which accelerates federated hyperparameter tuning by connecting to neural architecture search techniques. Examine theoretical foundations and empirical results demonstrating FedEx's superior performance on benchmarks like Shakespeare, FEMNIST, and CIFAR-10. Understand the importance of efficient hyperparameter tuning in federated learning and its impact on model accuracy and training budgets.

Syllabus

Introduction
Machine Learning
Outline
Hyperparameter Tuning Global vs Local
Hyperparameter Tuning Methods
Baseline Challenges
Success of Having
Issues
Resource Limitations
Local vs Global Validation
Baselines vs Bayesian Optimization
Neural Architecture Search
Architecture Search
Weight Sharing
Constraints
Federated Learning
Federated Averaging
Local Hyperparameters
Local Hyperparameter Tuning
Summary
Solution
Methods
Results
Key takeaways
Questions


Taught by

Stanford MedAI

Tags

Related Courses

Introduction to Artificial Intelligence
Stanford University via Udacity
Natural Language Processing
Columbia University via Coursera
Probabilistic Graphical Models 1: Representation
Stanford University via Coursera
Computer Vision: The Fundamentals
University of California, Berkeley via Coursera
Learning from Data (Introductory Machine Learning course)
California Institute of Technology via Independent