Towards General Purpose Vision: Insights from Multimodal AI Systems

Explore the evolution of multimodal AI systems in this insightful 20-minute conference talk from the Future Technologies Conference 2023. Gain valuable insights from Derek Hoiem of the University of Illinois at Urbana-Champaign as he delves into the development of advanced models capable of processing text, images, audio, and more. Discover the transformative impact of general-purpose models like GPT and their role in shaping AI's future. Learn about innovative approaches such as Unified IO and Visual Programming, which seamlessly integrate various modalities to solve complex tasks. Examine the challenges and breakthroughs in creating these systems, from specialized architectures to comprehensive memory systems. Through real-world examples and in-depth analysis, understand the potential of multimodal AI and the key to unlocking human-like intelligence in AI systems. Cover topics including custom NLP tasks, the differences between NLP and vision data, vision model evolution, AI limitations, and the importance of memory in AI systems.

Introduction
How to make a custom NLP task
NLP vs Vision data
Vision model evolution
AI limitations
Memory
Conclusion