A Recipe for Mixed Reality in Mario Kart Live

Explore the innovative fusion of physical and digital realms in Mario Kart Live: Home Circuit through this 53-minute GDC talk. Delve into the tools, techniques, and challenges faced by Velan Studios in creating a mixed reality racing experience. Learn about the four-gate system, driving mechanics, camera calibration, and data streams from IMU samples and video. Discover the intricacies of banner detection, transition searches, pose estimation, and SLAM (Simultaneous Localization and Mapping) methodologies. Gain insights into trackspace user experience, gate progress tracking, and dynamic content rendering. Uncover the secrets behind seamlessly integrating physical play spaces with digital game elements to bring Mario Kart to life in players' homes.

Intro
Four Gates
Driving Experience
Mixed Reality Racing
Physical Model - Play Space
Camera Calibration
Data Stream - IMU Sample
Data Stream - Video
Banner Detection
Search For Transitions
Walk Transitions
Pose Estimation
Marker Resources
SLAM Definition The computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent's location with it.
SLAM Methods
EKF Update
SLAM - Example Run
Predict With IMU
SLAM - Pure Integration
Measure - Stillness
Measure - Driving
SLAM - Gate Measurement
SLAM - Results
Trackspace - User Experience
Trackspace - Breadcrumbs
Trackspace Distance
Track Progress
Gate Progress
Move the Game Kart
Interpolate Toward SLAM
Place Gate Content
Update Gate
Missing Gate
Trackspace Items
Dynamic Content
Dynamic Items
Render Everything