Multimodal Generative AI: Technology Overview and Business Implications

Explore multimodal generative AI in this comprehensive 1-hour 38-minute conference talk from Microsoft Reactor Bengaluru. Delve into the technical aspects of training generative AI systems that handle multiple input types simultaneously, including text, image, and audio. Learn about business applications, limitations, and associated costs of these advanced systems. Gain insights into the open-source LLaVA (Large Language-and-Vision Assistant) multimodal system. Discover key concepts such as data gathering, outliers, nonlinearities, and the differences between statistics and AI. Examine practical examples like analog gauges and conversational systems. Understand the architecture, training data sets, and challenges like catastrophic forgetting and repeatability crisis. Investigate advanced topics including eigenvalue decomposition and visual inspection techniques. Access the accompanying presentation slides for a deeper understanding of the material covered.

Intro
What is a Mode
Why Everyone Gathers Data
Outliers
Assumption of Distribution
NonLinearities
Statistics vs AI
Vision vs Text
Netnet
Demo
Analog Gauges
Conversational Systems
Architecture
Example
How did they get the yes
Where did they get the data
Training data set
Mechanical Turk
Catastrophic forgetting
repeatability crisis
why I love this space
eigenvalue decomposition
Visual inspection