YoVDO

Associative Memories as a Building Block in Transformers

Offered By: Simons Institute via YouTube

Tags

Transformers Courses Machine Learning Courses Neural Networks Courses Computational Models Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore the role of associative memories in Transformer models through this 39-minute lecture by Alberto Bietti from the Flatiron Institute. Delve into the internal mechanisms of large language models and their ability to store vast amounts of knowledge from training data. Examine theoretical results on gradient-based learning of weight matrices as associative memories and the impact of over-parameterization on storage capacity. Gain insights into how Transformers adapt to new information in context or prompts through analysis of toy tasks for reasoning and factual recall. Enhance your understanding of Transformers as a computational model and their implications for reliable AI systems.

Syllabus

Associative memories as a building block in Transformers


Taught by

Simons Institute

Related Courses

Linear Circuits
Georgia Institute of Technology via Coursera
مقدمة في هندسة الطاقة والقوى
King Abdulaziz University via Rwaq (رواق)
Magnetic Materials and Devices
Massachusetts Institute of Technology via edX
Linear Circuits 2: AC Analysis
Georgia Institute of Technology via Coursera
Transmisión de energía eléctrica
Tecnológico de Monterrey via edX