YoVDO

CMU Advanced NLP: Pre-training Methods

Offered By: Graham Neubig via YouTube

Tags

Natural Language Processing (NLP) Courses BERT Courses Multi-Task Learning Courses Autoencoders Courses Contrastive Learning Courses Sentence Embedding Courses

Course Description

Overview

Explore advanced natural language processing techniques in this comprehensive lecture on pre-training methods. Delve into multi-task learning concepts, sentence embeddings, BERT and its variants, and alternative language modeling objectives. Gain insights into sentence representations, semantic similarity, textual entailment, and various pretraining approaches including autoencoders, skip-thought vectors, and paraphrase-based contrastive learning. Examine the impact of context and masking in language models, and understand the applications of these techniques in real-world NLP tasks.

Syllabus

Introduction
Neural Networks
Goals
Multitasking learning
Level of variety
Multitasking
Related Tasks
Multitask Learning
Pretraining
Pretraining Methods
Sentence Representations
Sentence Pair Classification
Sentence Pair Classification Examples
Semantic Similarity Relatedness
Textual entailment
Methods
Autoencoder
Skip thought vectors
Paraphrasebased contrastive learning
Largescale paraphrasing
Multitasking entailment
Supervised training
Sentence transformers
Context effect
Masking


Taught by

Graham Neubig

Related Courses

Structuring Machine Learning Projects
DeepLearning.AI via Coursera
Структурирование проектов по машинному обучению
DeepLearning.AI via Coursera
머신 러닝 프로젝트 구조화
DeepLearning.AI via Coursera
Stanford CS330: Deep Multi-Task and Meta Learning
Stanford University via YouTube
Stanford Seminar - The Next Generation of Robot Learning
Stanford University via YouTube