Word2Vec- Distributed Representations of Words and Phrases and Their Compositionality

Explore the influential Word2Vec technique for generating distributed word representations in this comprehensive video lecture. Delve into the Skip-Gram model, hierarchical softmax, and negative sampling methods. Learn about the mysterious 3/4 power, frequent word subsampling, and their impact on training efficiency. Examine empirical results and gain insights into the practical applications of Word2Vec in modern natural language processing. Understand how this technique captures syntactic and semantic word relationships, and discover its limitations in representing word order and idiomatic phrases.

- Intro & Outline
- Distributed Word Representations
- Skip-Gram Model
- Hierarchical Softmax
- Negative Sampling
- Mysterious 3/4 Power
- Frequent Words Subsampling
- Empirical Results
- Conclusion & Comments