From Robotics to AI NPCs - Embodied AI Agents in Games

Explore the convergence of robotics, simulation, and AI in a 26-minute talk by Nyla Worker, product lead at Convai. Discover how multi-modal Large Language Models power Non-Player Characters (NPCs) in games, drawing parallels between training AIs in simulated environments and creating dynamic virtual entities. Gain insights into Convai's methodologies for enabling NPCs to perform actions and react to their surroundings. Learn about embodied AI agents, perception, planning, and environmental feedback in the context of AI NPCs. Understand the applications of voice recognition, multiple languages, and human-computer interaction in this emerging field. Benefit from Worker's extensive experience in AI, product management, and deploying AI in production systems, including her work at NVIDIA's Omniverse Replicator and eBay's image recommendation system.

Intro
embodied AI agents
building a robot
building an AI NPC
Perception
Actions
Planning
Demonstration
Machine Learning AI
Voice Recognition
Multiple Languages
Embodied
Human Computer Interaction
Environmental Feedback
Conclusion
Questions