Building Data Infrastructure at Scale for AI/ML with Open Data Lakehouses

Explore how data lakehouse architecture with Apache Hudi supports real-world predictive ML and vector-based AI use cases in this 30-minute keynote by Vinoth Chandar, creator of Apache Hudi. Learn about ingesting data with minute-level freshness, providing a single source of truth for structured and unstructured data, and utilizing lakehouses for feature engineering, training dataset generation, and production feature creation. Discover the role of lakehouses in GenAI applications, including operating vector generation pipelines at scale and integrating with vector databases for real-time serving. Gain insights from use cases across organizations like NielsenIQ, Notion, and Uber, and understand how data engineers can leverage existing tools or develop new solutions to address AI and ML challenges.

Building Data Infrastructure at Scale for AI/ML with Open Data Lakehouses // Vinoth Chandar // DE4AI