CMU Advanced NLP: Pre-training Methods
Offered By: Graham Neubig via YouTube
Course Description
Overview
Explore advanced natural language processing techniques in this comprehensive lecture on pre-training methods. Delve into multi-task learning concepts, sentence embeddings, BERT and its variants, and alternative language modeling objectives. Gain insights into sentence representations, semantic similarity, textual entailment, and various pretraining approaches including autoencoders, skip-thought vectors, and paraphrase-based contrastive learning. Examine the impact of context and masking in language models, and understand the applications of these techniques in real-world NLP tasks.
Syllabus
Introduction
Neural Networks
Goals
Multitasking learning
Level of variety
Multitasking
Related Tasks
Multitask Learning
Pretraining
Pretraining Methods
Sentence Representations
Sentence Pair Classification
Sentence Pair Classification Examples
Semantic Similarity Relatedness
Textual entailment
Methods
Autoencoder
Skip thought vectors
Paraphrasebased contrastive learning
Largescale paraphrasing
Multitasking entailment
Supervised training
Sentence transformers
Context effect
Masking
Taught by
Graham Neubig
Related Courses
Sentiment Analysis with Deep Learning using BERTCoursera Project Network via Coursera Natural Language Processing with Attention Models
DeepLearning.AI via Coursera Fine Tune BERT for Text Classification with TensorFlow
Coursera Project Network via Coursera Deploy a BERT question answering bot on Django
Coursera Project Network via Coursera Generating discrete sequences: language and music
Ural Federal University via edX