YoVDO

Neural Nets for NLP 2021 - Conditioned Generation

Offered By: Graham Neubig via YouTube

Tags

Neural Networks Courses Natural Language Processing (NLP) Courses Search Algorithms Courses Encoder-Decoder Models Courses

Course Description

Overview

Learn about conditioned generation in neural networks for natural language processing in this lecture from CMU's Neural Networks for NLP course. Explore encoder-decoder models, conditional generation techniques, and search algorithms like beam search. Examine methods for ensembling multiple models, including linear interpolation and parameter averaging. Discover various types of data that can be used to condition language models, from speaker traits to word embeddings. Gain insights into evaluation paradigms for generative models, covering human evaluation, embedding-based metrics, and perplexity. Understand the strengths and limitations of different evaluation approaches for assessing language model performance.

Syllabus

Intro
Language Models • Language models are generative models of text
Conditioned Language Models
Calculating the Probability of a Sentence
Conditional Language Models
One Type of Language Model Mikolov et al. 2011
How to Pass Hidden State?
The Generation Problem
Ancestral Sampling
Greedy Search
Beam Search
Ensembling . Combine predictions from multiple models
Linear Interpolation • Take a weighted average of the M model probabilities
Log-linear Interpolation • Weighted combination of log probabilities, normalize
Linear or Log Linear?
Parameter Averaging
Ensemble Distillation (e.g. Kim et al. 2016)
Stacking
Still a Difficult Problem!
From Speaker/Document Traits (Hoang et al. 2016)
From Lists of Traits (Kiddon et al. 2016)
From Word Embeddings (Noraset et al. 2017)
Basic Evaluation Paradigm
Human Evaluation Shared Tasks
Embedding-based Metrics
Perplexity
Which One to Use?


Taught by

Graham Neubig

Related Courses

TensorFlow Developer Certificate Exam Prep
A Cloud Guru
Post Graduate Certificate in Advanced Machine Learning & AI
Indian Institute of Technology Roorkee via Coursera
Advanced AI Techniques for the Supply Chain
LearnQuest via Coursera
Advanced Learning Algorithms
DeepLearning.AI via Coursera
IBM AI Engineering
IBM via Coursera