Associative Memories as a Building Block in Transformers
Offered By: Simons Institute via YouTube
Course Description
Overview
Explore the role of associative memories in Transformer models through this 39-minute lecture by Alberto Bietti from the Flatiron Institute. Delve into the internal mechanisms of large language models and their ability to store vast amounts of knowledge from training data. Examine theoretical results on gradient-based learning of weight matrices as associative memories and the impact of over-parameterization on storage capacity. Gain insights into how Transformers adapt to new information in context or prompts through analysis of toy tasks for reasoning and factual recall. Enhance your understanding of Transformers as a computational model and their implications for reliable AI systems.
Syllabus
Associative memories as a building block in Transformers
Taught by
Simons Institute
Related Courses
Linear CircuitsGeorgia Institute of Technology via Coursera مقدمة في هندسة الطاقة والقوى
King Abdulaziz University via Rwaq (رواق) Magnetic Materials and Devices
Massachusetts Institute of Technology via edX Linear Circuits 2: AC Analysis
Georgia Institute of Technology via Coursera Transmisión de energía eléctrica
Tecnológico de Monterrey via edX