Generalization in Diffusion Models from Geometry-Adaptive Harmonic Representation
Offered By: Valence Labs via YouTube
Course Description
Overview
Explore the intricacies of generalization in diffusion models through a comprehensive lecture on geometry-adaptive harmonic representation. Delve into the world of AI for drug discovery as Zahra Kadkhodaie from Valence Labs presents a detailed analysis of how deep neural networks (DNNs) learn high-dimensional densities despite the curse of dimensionality. Examine the strong generalization capabilities of denoising DNNs and their alignment with data distribution properties. Investigate the shrinkage operation performed by denoisers in an image-adapted basis, revealing oscillating harmonic structures along contours and in homogeneous regions. Discover how trained denoisers are inductively biased towards geometry-adaptive harmonic representations, even when trained on suboptimal image classes. Gain insights into the near-optimal denoising performance of networks trained on regular image classes. The lecture covers various topics, including diffusion models, denoising, the transition from memorization to generalization, denoising as shrinkage in a basis, and inductive biases, concluding with a Q&A session.
Syllabus
- Intro + Background
- Diffusion Models + Denoising
- Transition from Memorization to Generalization
- Denoising as Shrinkage in a Basis
- Inductive Biases
- Q + A
Taught by
Valence Labs
Related Courses
Introduction to Artificial IntelligenceStanford University via Udacity Natural Language Processing
Columbia University via Coursera Probabilistic Graphical Models 1: Representation
Stanford University via Coursera Computer Vision: The Fundamentals
University of California, Berkeley via Coursera Learning from Data (Introductory Machine Learning course)
California Institute of Technology via Independent