Alternatives to Reinforcement Learning for Real-World Problems

Explore alternatives to Reinforcement Learning for real-world problems in this 42-minute conference talk from Open Data Science. Delve into the limitations of Reinforcement Learning in practical applications, focusing on the challenges of simulation and full observability. Discover two related approaches for agent-based learning: Contextual Bandits and Imitation Learning. Learn how these methods simplify the full Reinforcement Learning problem, their formal definitions, differences, limitations, and real-world applications. Gain insights into tools like Microsoft Azure and AWS SageMaker, and understand when to use each approach. Examine concepts such as behavioral cloning, expert systems, and interactive experts. Explore the scalability concerns, data capture methods, and the exciting potential of combining Imitation Learning with Reinforcement Learning. Conclude with a discussion on Offline RL and its significance in addressing real-world challenges.

Intro
LET'S TALK ABOUT REINFORCEMENT LEARNING
THE THREE MACHINE LEARNS
EMBODIED LEARNING
AGENT-BASED LEARNING
THE DECISION POLICY
THE REWARD
TWO IDEAS
DEALING WITH UNCERTAINTY
REQUIREMENTS OF BIG SUCCESSES
SIMULATION
FULLY OBSERVABLE
TRANSFERABILITY OF METHOD
WHAT IS THE COST OF AN ERROR?
CAN WE APPLY THIS TO REAL PROBLEMS?
REAL-WORLD ALTERNATIVES
WHAT ARE WE TRYING TO SOLVE
TOOLS
MICROSOFT AZURE
AWS SAGEMAKER
WHEN SHOULD I USE CONTEXTUAL BANDITS?
LIMITATIONS
BEHAVIORAL CLONING
EXPERT SYSTEMS SUPERVISED LEARNING
COLLECT TRAJECTORIES FROM AN EXPERT
BREAK UP INTO STATE / ACTION PAIRS
TRAIN A MODEL ON THE TRAJECTORIES
INTERACTIVE EXPERTS
APPLICATIONS
WHEN SHOULD I USE IMITATION LEARNING?
SCALABILITY CONCERNS
CAPTURING DATASETS
IMITATION LEARNING + REINFORCEMENT LEARNING
RESOURCES
OFFLINE RL
WHY IS THIS EXCITING?