YoVDO

Offline Reinforcement Learning and Model-Based Optimization

Offered By: Simons Institute via YouTube

Tags

Offline Reinforcement Learning Courses Predictive Modeling Courses

Course Description

Overview

Explore offline reinforcement learning and model-based optimization in this 34-minute lecture by Sergey Levine from UC Berkeley. Delve into the power of predictive models and automated decision-making, focusing on data-driven reinforcement learning and model-based optimization. Learn about off-policy RL, distribution shift challenges, and Q-function lower bounds. Examine the CQL algorithm and its performance. Investigate predictive modeling and design, addressing issues with simple prediction and exploring model-based optimization problems. Discover uncertainty and extrapolation concepts, and understand model inversion networks (MINS). Analyze experimental results and gain valuable insights into these cutting-edge machine learning techniques.

Syllabus

Intro
What makes modern machine learning word
Predictive models are very powerful!
Automated decision making is very powerf
First setting: data-driven reinforcement lear
Second setting: data-driven model-based optimization
Off-policy RL: a quick primer
What's the problem?
Distribution shift in a nutshell
How do prior methods address this?
Learning with Q-function lower bounds Algorithm
Does the bound hold in practice?
How does CQL compare?
Predictive modeling and design
What's wrong with just doing prediction?
The model-based optimization problem
Uncertainty and extrapolation
What can we do?
Model inversion networks (MINS)
Putting it all together
Experimental results
Some takeaways
Some concluding remarks


Taught by

Simons Institute

Related Courses

Can Wikipedia Help Offline Reinforcement Learning - Author Interview
Yannic Kilcher via YouTube
Can Wikipedia Help Offline Reinforcement Learning? - Paper Explained
Yannic Kilcher via YouTube
CAP6412 - Final Project Presentations - Lecture 27
University of Central Florida via YouTube
Reinforcement Learning
Simons Institute via YouTube
What Are the Statistical Limits of Offline Reinforcement Learning With Function Approximation?
Simons Institute via YouTube