Feedback Control from Pixels

Explore feedback control using camera inputs in this MIT Embodied Intelligence Seminar featuring Russ Tedrake, Toyota Professor at MIT. Delve into the challenges of integrating control theory with visual feedback, examining recent advancements in reinforcement learning and imitation learning. Learn about attempts to bridge the gap between camera-based control and traditional systems theory, with Tedrake presenting recent results and small steps towards this goal. Gain insights into topics such as training correspondences, key point affordances, model-based policy search, and linear ARX models. Discover how these concepts apply to robotics, including discussions on robot representation and the diversity of tasks in feedback control from pixels.

Introduction
Welcome
Feedback control from pixels
Training correspondences
Language of control
Cameras
Limitations
Output feedback problem
Fundamental problem
General framework
State representation
Key point affordances
Key point base affordances
Modelbased policy search
Parameterizations
Reinforcement learning
Output feedback
RX models
State feedback
Linear models
Linear ARX models
A new problem
Carrots
Objective
Image
Image coordinates
Learning a model
Simple thought experiment
Action frame
Least squares
Linear map
Preimage
Closed loop performance
Next steps
Questions
Linear prediction
Robot representation
Diversity of tasks