Neural Nets for NLP 2017 - Adversarial Learning

Explore adversarial learning in neural networks for natural language processing in this lecture from CMU's CS 11-747 course. Dive into generative adversarial networks (GANs), examining their application to both features and outputs in NLP tasks. Learn about the challenges of using GANs with discrete outputs and discover techniques for overcoming these obstacles. Investigate the use of adversarial training methods for domain adaptation, multi-task learning, and unsupervised style transfer in text. Gain insights into stabilization tricks and innovative applications like GAN-based data cleaning. Enhance your understanding of advanced NLP concepts through practical examples and theoretical explanations provided by Professor Graham Neubig.

Intro
Generative Models
Adversarial Training
Basic Paradigm
Problems with Generation • Over-emphasis of common outputs, fuzziness Adversarial
Training Method
In Equations
Problems w/ Training
Applications of GAN Objectives to Language
Problem! Can't Backprop through Sampling
Solution: Use Learning Methods for Latent Variables
Discriminators for Sequences
Stabilization Trick
Interesting Application: GAN for Data Cleaning (Yang et al. 2017)
Adversaries over Features vs. Over Outputs
Learning Domain-invariant Representations (Ganin et al. 2016) • Learn features that cannot be distinguished by domain
Adversarial Multi-task Learning (Liu et al. 2017)
Implicit Discourse Connection Classification w/ Adversarial Objective
Professor Forcing (Lamb et al. 2016)
Unsupervised Style Transfer for Text (Shen et al. 2017)