Towards Generalization of Precise Robot Skills - Accurate Pick-and-Place of Novel Objects

Explore a comprehensive approach to robotic pick-and-place that combines high precision and generalization skills. Learn how simulation-based probabilistic models for grasping, planning, and localization can be effectively transferred to real robotic systems. Discover the capabilities of a dual-arm robot in exerting task-aware picks on new objects, utilizing visuo-tactile sensing for localization, and performing dexterous placings with sub-millimeter accuracy. Gain insights into achieving precise robotic generalization through learned probabilistic models, enabling robots to adapt their skills across multiple tasks. Delve into the research of Maria Bauza Villalonga, a PhD student at MIT, as she presents her work on advancing robotic capabilities in handling novel objects and placing configurations with high precision.

Intro
Current progress toward generalization lacks precision
My goal: precise robotic generalization
What do we need to solve precise pick-and-place?
Test time: pick-and-place on the real system
What do we mean by tactile localization?
Pipeline for tactile pose estimation
What does it mean that 2 images are close?
We compute pose error using the object's pointcloud
Tactile localization using contrastive learning
Quantitative results on tactile localization
Pipeline for visual pose estimation
Benefits of matching images with encodings
Our solutions allows different goal configurations
Lessons learned and future directions