CS480-680 Lecture 19 - Attention and Transformer Networks

Explore the fundamental concepts of attention mechanisms and transformer networks in this comprehensive lecture. Delve into topics such as attention neural networks, kernel similarity, and machine translation. Gain insights into the architecture of transformer networks, including multihead attention and mask multihead attention. Examine the role of recurrence and normalization in these advanced deep learning models. Enhance your understanding of cutting-edge natural language processing techniques and their applications in various domains.

Intro
Attention
Attention Neural Networks
Kernel Similarity
Machine Translation
Transformer Networks
Multihead Attention
Mask Multihead Attention
Recurrence
Normalization