YoVDO

Attention and the Transformer

Offered By: Alfredo Canziani via YouTube

Tags

Transformer Architecture Courses Deep Learning Courses Neural Networks Courses PyTorch Courses Attention Mechanisms Courses

Course Description

Overview

Explore the intricacies of attention mechanisms and the Transformer architecture in this comprehensive 1-hour 18-minute lecture by Alfredo Canziani. Delve into the concept of self-attention and its role in creating hidden layer representations of inputs. Discover the key-value store paradigm and learn how to represent queries, keys, and values as rotations of an input. Gain insights into the Transformer architecture through a detailed walkthrough of a forward pass and compare the encoder-decoder paradigm with sequential architectures. The lecture concludes with a Q&A session, providing an opportunity to clarify complex concepts and deepen understanding of attention mechanisms and their implementation in PyTorch.

Syllabus

– Week 12 – Practicum
– Attention
– Key-value store
– Transformer and PyTorch implementation
– Q&A


Taught by

Alfredo Canziani

Tags

Related Courses

Neural Networks for Machine Learning
University of Toronto via Coursera
Good Brain, Bad Brain: Basics
University of Birmingham via FutureLearn
Statistical Learning with R
Stanford University via edX
Machine Learning 1—Supervised Learning
Brown University via Udacity
Fundamentals of Neuroscience, Part 2: Neurons and Networks
Harvard University via edX