YoVDO

Scaling Insights from Infinite-Width Theory for Next Generation Architectures and Learning Paradigms

Offered By: Institute for Pure & Applied Mathematics (IPAM) via YouTube

Tags

Machine Learning Courses Neural Networks Courses Scaling Courses Transformers Courses Multilayer Perceptrons Courses ResNet Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore a comprehensive lecture on scaling insights derived from infinite-width theory for advanced architectures and learning paradigms. Delve into the critical role of scaling in modern machine learning and its associated challenges, such as increased training instability. Discover how infinite-width theory can be applied to establish optimal scaling rules across various architectures and learning paradigms. Examine the scaling behavior of Multilayer Perceptrons (MLPs) under Sharpness-Aware Minimization, a min-max learning formulation designed to enhance generalization. Investigate how this analysis extends to other architectures like transformers, ResNets, and CNNs. Learn about the unique scaling behavior of structured state space models (SSMs), which have emerged as efficient alternatives to transformers. Understand the need for specialized approaches in analyzing SSMs due to their unique transition matrix structure. Gain insights into the scaling of SSMs within the standard minimization framework and explore the implications of specialized scaling strategies.

Syllabus

Leena Vankadara - Scaling Insights from Infinite-Width Theory for Next Gen Architecture & Learning


Taught by

Institute for Pure & Applied Mathematics (IPAM)

Related Courses

Creating Multi Task Models With Keras
Coursera Project Network via Coursera
Deep Learning: Advanced Computer Vision (GANs, SSD, +More!)
Udemy
【Hands Onで学ぶ】PyTorchによる深層学習入門
Udemy
Axial-DeepLab - Stand-Alone Axial-Attention for Panoptic Segmentation
Yannic Kilcher via YouTube
Deep Residual Learning for Image Recognition - Paper Explained
Yannic Kilcher via YouTube