Successes and Challenges in Neural Models for Speech and Language - Michael Collins

Explore the evolution and challenges of neural models in speech and language processing through this insightful lecture by Michael Collins from Google Research and Columbia University. Delve into the statistical and neural revolutions in natural language processing, examining key concepts such as kernel methods, word embeddings, and parsing problems. Learn about innovative architectures like Transformers and Multi-Head Transformers, and their applications in solving complex language tasks. Gain a comprehensive understanding of three significant problems in the field and the corresponding architectures designed to address them.

Intro
Problems in Speech and Natural Language
The First (Statistical) Revolution
The Second (Neural) Revolution
A Personal View: the Parsing Problem
Kernel Methods
Word Embeddings
Natural Language Syntax, and the Parsing Problem
Shift Actions
Predicting Actions
The Natural Questions Data
Transformers (continued)
Multi-Head Transformers
This Talk: Three Problems, Three Architectures