Neural Nets for NLP 2019 - Convolutional Neural Networks for Language

Explore convolutional neural networks for natural language processing in this comprehensive lecture from CMU's Neural Networks for NLP course. Dive into bag of words and n-grams concepts before examining various applications of convolution in language modeling. Learn about context windows, sentence modeling, and structured convolution techniques. Discover the power of stacked and dilated convolutions in processing linguistic data. Investigate convolutional models for analyzing sentence pairs. Gain insights into pooling strategies, non-linear activation functions, and their impact on language processing tasks. Enhance your understanding of advanced CNN architectures tailored for NLP applications through detailed explanations and practical examples.

Intro
A First Try: Bag of Words (BOW)
Continuous Bag of Words (CBOW) this movie
What do Our Vectors Represent?
Bag of n-grams hate
Why Bag of n-grams?
2-dimensional Convolutional Networks
CNNs for Sentence Modeling
Standard conv2d Function
Padding
Striding
Pooling . Pooling is like convolution, but calculates some reduction function feature-wise • Max pooling: "Did you see this feature anywhere in the range?" (most common) • Average pooling: How prevalent is this feature over the entire range
Stacked Convolution
Dilated Convolution (e.g. Kalchbrenner et al. 2016) . Gradually increase stride every time step (na reduction in length) sentence
Iterated Dilated Convolution (Strubell+2017) . Multiple iterations of the same stack of dilated convolutions
Non-linear Functions
Which Non-linearity Should I Use?