Random Matrices and Dynamics of Optimization in Very High Dimensions - Lecture 2

Explore a comprehensive lecture on random matrices and optimization dynamics in high-dimensional spaces. Delve into the complexities of machine learning and data science algorithms, focusing on the effectiveness of simple tools like Stochastic Gradient Descent in complex, over-parametrized regimes. Gain insights into the framework of typical tasks and neural network structures used in standard contexts. Examine the classical context of SGD in finite dimensions before surveying recent work on projected "effective dynamics" for summary statistics in smaller dimensions. Investigate how these dynamics govern the performance of high-dimensional systems and define complex finite-dimensional dynamical systems. Discover the process of identifying summary statistics through a dynamical spectral transition in Random Matrix Theory, exploring the behavior of Gram and Hessian matrices along optimization paths. Apply these concepts to central machine learning examples, including multilayer neural networks for classification of Gaussian mixtures and XOR problems. Enhance your understanding of random matrix tools, particularly the behavior of spectrum edges and the BBP transition, in a broader context.

Gérard Ben Arous - 2/4 Random Matrices and Dynamics of Optimization in Very High Dimensions