The Mathematical Foundations of Deep Learning: From Rating Impossibility to Practical Existence Theorems

Explore the mathematical foundations of deep learning in this comprehensive lecture by Simone Brugiapaglia from Concordia University. Delve into two case studies: rating impossibility theorems in cognitive science applications and practical existence theorems in scientific computing. Examine the limitations of deep learning in generalizing outside training sets for identity effect classification tasks. Discover how universal approximation results for deep neural networks combine with compressed sensing and high-dimensional polynomial approximation theory to yield sufficient conditions for accurate function approximation. Gain insights into ongoing research and open questions in the field, covering topics such as AI subfields, computational mathematics, deep neural networks, and approximation techniques. Enhance your understanding of the mathematical challenges and potential advancements in deep learning through this in-depth presentation.

Introduction
Collaborators
AI Index Report
AI Subfields
Impact of Deep Learning
Computational Mathematics and Deep Learning
Deep Learning Skepticism
Mathematical Problems for the Next Century
Presentation Structure
Deep Neural Networks
Research Question
Can Deep Learning generalize
The connectionist
Notation
General Results
Tau
Training History
Approximation
Approximation with orthogonal polynomials
Approximation techniques
In practice
Compressed sensing
Recap
Research directions
Conclusion