CMU Advanced NLP 2021 - Latent Variable Models

Explore latent variable models in advanced natural language processing through this comprehensive lecture from CMU's CS 11-711 course. Delve into the distinctions between generative and discriminative models, as well as deterministic and random variables. Gain insights into Variational Autoencoders (VAEs) and their applications in NLP, including techniques for handling discrete latent variables. Examine the trade-offs between learning features and learning structure. Cover topics such as evidence lower bound, procedural training, KL divergence annealing, and aggressive inference network learning. Understand sampling methods and reparameterization for discrete latent variables, equipping you with advanced knowledge for implementing sophisticated NLP models.

Introduction
Discriminative vs generative models
Types of variables
Loss function
Two tasks
Bias and variance
Evidence lower bound
Procedural training
Questions
Learning the VAE
Generating Sentences
Problems
kl divergence annealing
Free bits
Weaken the decoder
Aggressive inference network learning
Standard variational autoencoder
What are discrete latent variables
Method 1 Sampling
Method 2 Sampling
Method 2 Reparameterization