Transforming Autonomous Driving with Large Language Models and Generative AI

Explore the cutting-edge developments in autonomous driving technology through a 29-minute conference talk by Remi Tachet des Combes, Senior Applied Scientist at Wayve. Dive into the innovative LINGO and GAIA models, which leverage Large Language Models and Generative AI to revolutionize self-driving vehicles. Discover how LINGO, an open-loop driving commentator, enhances safety communication and addresses model hallucinations through its unique "show and tell" feature and referential segmentation. Learn about GAIA, an advanced generative world model that simulates realistic driving scenarios, improving decision-making and safety in autonomous vehicles. Gain insights into the integration of vision, language, and action in Vision-Language-Action Models (VLAMs) and their potential for human-like communication in AV technology. Understand the scalability and superior video generation quality of GAIA's 6.5 billion parameter autoregressive transformer, trained on extensive driving data. Explore the speaker's background in applied mathematics, reinforcement learning, and deep learning, and his current focus on world modeling and representation learning for autonomy at Wayve.

Introduction
Context
Modern AI
Embodied AI
Research Highlights
World Models
Questions