Emergence and Grokking in Simple Neural Architectures
Offered By: Institute for Pure & Applied Mathematics (IPAM) via YouTube
Course Description
Overview
Explore a thought-provoking lecture on the emergence and grokking phenomena in simple neural network architectures. Delve into Misha Belkin's presentation at IPAM's Theory and Practice of Deep Learning Workshop, where he argues that Multi-Layer Perceptrons (MLPs) exhibit remarkable behaviors similar to those observed in modern Large Language Models. Examine the challenges in understanding how 2-layer MLPs learn relatively simple problems like "grokking" modular arithmetic. Discover recent progress in the field and learn about Recursive Feature Machines as a potential model for analyzing emergent phenomena in neural architectures. Gain valuable insights into the computational aspects of modern neural networks and their implications for deep learning theory and practice.
Syllabus
Misha Belkin - Emergence and grokking in "simple" architectures - IPAM at UCLA
Taught by
Institute for Pure & Applied Mathematics (IPAM)
Related Courses
Neural Networks for Machine LearningUniversity of Toronto via Coursera Good Brain, Bad Brain: Basics
University of Birmingham via FutureLearn Statistical Learning with R
Stanford University via edX Machine Learning 1—Supervised Learning
Brown University via Udacity Fundamentals of Neuroscience, Part 2: Neurons and Networks
Harvard University via edX