Transformer Encoder in 100 Lines of Code
Offered By: CodeEmporium via YouTube
Course Description
Overview
Syllabus
What we will cover
Introducing Colab
Word Embeddings and d_model
What are Attention heads?
What is Dropout?
Why batch data?
How to sentences into the transformer?
Why feed forward layers in transformer?
Why Repeating Encoder layers?
The “Encoder” Class, nn.Module, nn.Sequential
The “EncoderLayer” Class
What is Attention: Query, Key, Value vectors
What is Attention: Matrix Transpose in PyTorch
What is Attention: Scaling
What is Attention: Masking
What is Attention: Softmax
What is Attention: Value Tensors
CRUX OF VIDEO: “MultiHeadAttention” Class
Returning the flow back to “EncoderLayer” Class
Layer Normalization
Returning the flow back to “EncoderLayer” Class
Feed Forward Layers
Why Activation Functions?
Finish the Flow of Encoder
Conclusion & Decoder for next video
Taught by
CodeEmporium
Related Courses
Neural Networks for Machine LearningUniversity of Toronto via Coursera 機器學習技法 (Machine Learning Techniques)
National Taiwan University via Coursera Machine Learning Capstone: An Intelligent Application with Deep Learning
University of Washington via Coursera Прикладные задачи анализа данных
Moscow Institute of Physics and Technology via Coursera Leading Ambitious Teaching and Learning
Microsoft via edX