Generalization and Personalization in Federated Learning - Karan Singhal

Explore federated learning, generalization, and personalization in medical AI through this comprehensive lecture by Karan Singhal from Stanford University. Delve into two recent works on generalization in federated learning and federated reconstruction, examining their applications in medical settings. Learn about challenges in generalizing to unseen patients and hospitals, both in federated and centralized environments. Discover novel approaches to improve generalization across multiple local data distributions, and gain insights into representation learning techniques that can lead to wider adoption of beneficial AI in healthcare. Engage with concepts such as participation gaps, semantic partitioning, and client-specific embeddings, while understanding their implications for medical data analysis and AI deployment in clinical settings.

Introduction
Outline
Federated Learning
Client Devices
Federal Learning
Validation
Example
Characteristics of Federated Learning
Questions
Generalization
Generalization Gaps
Participation Gaps
Does Participation Gap exist
Different ways of making federated data sets
Natural vs labelbased partitioning
Semantic partitioning
Intuition
Results
MNIST
Generalization in MedAI
Distribution of Medical Data
Hospitals and Patients
Conclusions
Extending the 3way split
Takeaways
Next Part
Recap
Can we do better
Use case
Factorization
ClientSpecific Embedding
Local Stateful Embedding
Problems with Statefulness
Generalization in Federated Learning
Federal Reconstruction
Metal Learning
Next Word Prediction
Deployment
Takeaway
Preliminary results
Multilevel assumptions
Resources
Audience Questions