Augmenting Augmented Reality with Non-Line-of-Sight Perception

Explore a groundbreaking conference talk on X-AR, an innovative augmented reality (AR) system with non-line-of-sight perception capabilities. Discover how this technology enhances AR headsets with RF sensing, allowing users to perceive objects invisible to the human eye and current AR systems. Learn about the three key innovations: an AR-conformal antenna, an RF-visual synthetic aperture localization algorithm, and an RF-visual verification primitive. Delve into the implementation and evaluation of X-AR integrated into a Microsoft Hololens 2 AR headset, showcasing its ability to achieve decimeter-level RF localization of fully-occluded items and perform RF-visual picking verification with high accuracy. Gain insights into the potential applications of this technology in manufacturing, warehousing, and smart home environments, and witness its capabilities through a demonstration video.

NSDI '23 - Augmenting Augmented Reality with Non-Line-of-Sight Perception