Generative Artificial Intelligence for Decision Making in Robotics

Explore the application of generative AI in robotics through this insightful conference talk. Discover how video foundation models pretrained on internet data can serve as universal policies for environments with varying state and action spaces, enabling combinatorial generalization to unseen goals using text-based specifications. Learn about the composition of multiple foundation models pretrained on different internet data modalities to hierarchically construct physically executable plans for long-horizon robotic tasks. Examine the effectiveness and adaptability of this approach in various long-horizon table-top manipulation scenarios. Gain valuable insights into overcoming data limitations in robotics and leveraging generative AI advancements for improved decision-making in robotic systems.

Yilun Du and Anurag Ajay: Generative Artificial Intelligence for Decision Making