Neural Nets for NLP 2020 - Convolutional Neural Networks for Text

Explore convolutional neural networks for text processing in this comprehensive lecture from CMU's Neural Networks for NLP course. Delve into bag-of-words models, convolution applications for context windows and sentence modeling, advanced techniques like stacked and dilated convolutions, and structured convolution. Examine various applications of convolutional models in natural language processing and consider the inductive biases they introduce. Learn about sentiment classification, continuous bag of words, deep CBOW, and neural sequence models. Understand key concepts such as 2D convolution, stride, padding, multiple filters, and pooling. Discover the architecture of CNN models for NLP tasks, including embedding layers, convolutional layers, pooling layers, and output layers. Investigate specific applications like dynamic filter CNNs, NLP from scratch, and CNN-RNN-CRF models for tagging. Gain insights into the design philosophy and structural biases of convolutional models for text processing.

Intro
Outline
An Example Prediction Problem: Sentiment Classification
Continuous Bag of Words (CBOW)
Deep CBOW
Why Bag of n-grams?
What Problems
Neural Sequence Models
Definition of Convolution
Intuitive Understanding
Priori Entailed by CNNS
Concept: 2d Convolution
Concept: Stride
Concept: Padding
Three Types of Convolutions
Concept: Multiple Filters
Concept: Pooling
Overview of the Architecture
Embedding Layer
Conv. Layer
Pooling Layer
Output Layer
Dynamic Filter CNN (e.g. Brabandere et al. 2016)
CNN Applications
NLP (Almost) from Scratch (Collobert et al. 2011)
CNN-RNN-CRF for Tagging (Ma et al. 2016) . A classic framework and de-facto standard for
Why Structured Convolution?
Understand the design philosophy of a model
Structural Bias
component entail?