Building a Unified Feature Platform with DuckDB and Arrow

Dive into a technical talk by Mike Eastham, Chief Architect at Tecton, exploring the development of Tecton's innovative Feature Platform for Machine Learning. Learn how DuckDB and Arrow are leveraged to address data transformation challenges and provide a faster, more integrated local development experience. Discover effective strategies for scaling datasets without a distributed query engine, techniques for implementing DuckDB extensions, and insights on creating interoperability with DeltaLake outside the Spark ecosystem. Compare performance with Spark and understand how Tecton utilizes cloud resources to handle large datasets while maintaining an optimal laptop developer experience. Gain valuable knowledge on building cutting-edge data and AI systems from this 36-minute session presented at Data Council.

Building a Unified Feature Platform with DuckDB and Arrow