BLEURT - Learning Robust Metrics for Text Generation
Offered By: Yannic Kilcher via YouTube
Course Description
Overview
Explore a comprehensive video explanation of the BLEURT paper, which proposes a learned evaluation metric for text generation models. Dive into the challenges of evaluating machine translation systems and learn how BLEURT addresses these issues through a novel pre-training scheme using synthetic data. Discover the key components of the approach, including fine-tuning BERT, generating synthetic data, and priming via auxiliary tasks. Examine the experimental results, distribution shifts, and potential concerns associated with this innovative metric. Gain insights into the state-of-the-art performance of BLEURT on recent WMT Metrics shared tasks and the WebNLG Competition dataset.
Syllabus
- Intro & High-Level Overview
- The Problem with Evaluating Machine Translation
- Task Evaluation as a Learning Problem
- Naive Fine-Tuning BERT
- Pre-Training on Synthetic Data
- Generating the Synthetic Data
- Priming via Auxiliary Tasks
- Experiments & Distribution Shifts
- Concerns & Conclusion
Taught by
Yannic Kilcher
Related Courses
3D Deep Learning for Gaming with Srinath Sridhar and Stanford Artificial IntelligenceResemble AI via YouTube Deep Learning in Gaming with Idan Beck
Resemble AI via YouTube Preserving Patient Safety as AI Transforms Clinical Care - Curt Langlotz, Stanford University
Alan Turing Institute via YouTube Synthesizing Plausible Privacy-Preserving Location Traces
IEEE via YouTube Hacking the Pentagon - How a Rebel Alliance Shifts Culture to Protect National Security
BSidesLV via YouTube