Metric Space Magnitude and Generalization in Neural Networks

Explore the concept of metric space magnitude and its applications in neural network generalization in this 47-minute talk by Rayna Andreeva. Delve into the recently established invariant that measures the 'effective size' of a space across multiple scales. Discover how magnitude encodes many known invariants of a metric space and learn about its current applications in machine learning. Focus on the recent development of magnitude in deep learning, examining internal representations of neural networks and new topological complexity measures for determining generalization capabilities. Understand the computationally friendly algorithms proposed for calculating generalization indices and how this flexible framework can be extended to various domains, tasks, and architectures. Examine experimental results demonstrating high correlation between the new complexity measures and generalization error in industry-standard architectures like transformers and deep graph networks. Compare the approach to existing topological bounds across diverse datasets, models, and optimizers, highlighting its practical relevance and effectiveness in the field of applied algebraic topology and machine learning.

Rayna Andreeva (07/17/2024): Metric space magnitude and generalization in neural networks