Neural Nets for NLP 2018 - Models of Dialogue

Explore models of dialogue in this comprehensive lecture from CMU's Neural Nets for NLP 2018 course. Delve into various types of dialogue, generation-based models, and neural approaches for dialogue response generation. Learn about the importance of global coherence in dialogue and solutions like using standard architectures with more context. Examine discourse-level VAE models, diversity-promoting objectives, and methods for evaluating dialogue systems. Discover techniques for infusing personality into dialogue agents and explore retrieval-based chat systems. Gain insights into neural approaches for natural language understanding, slot filling, dialog state tracking, and language generation from dialog states.

Types of Dialog
Two Paradigms
Generation-based Models (Ritter et al. 2011)
Neural Models for Dialog Response Generation
Dialog More Dependent on Global Coherence
One Solution: Use Standard Architecture w/ More Context
Discourse-level VAE Model (Zhao et al. 2017)
Diversity Promoting Objective for Conversation (Li et al. 2016) • Basic idea we want responses that are likely given the context, unlikely otherwise • Method: subtract weighted unconditioned log probability from conditioned probability (calculated only on first few words)
Using Multiple References with Human Evaluation Scores (Gallay et al. 2015)
Learning to Evaluate • Use context, true response, and actual response to learn a regressor that predicts goodness (Lowe et al. 2017)
Problem 3: Dialog Agents should have Personality
Personality Infused Dialog (Mairesse et al. 2007)
Dialog Response Retrieval
Retrieval-based Chat (Lee et al. 2009)
Neural Response Retrieval (Nio et al. 2014)
Smart Reply for Email Retrieval (Kannan et al. 2016)
NLU (for Slot Filling) w/ Neural Nets (Mesnil et al. 2015)
Dialog State Tracking
Language Generation from Dialog State w/ Neural Nets (Won et al. 2015)