PandasUDFs - Scaling Ensembles for Improved Predictions

Discover how to leverage PandasUDFs as a powerful technique for scaling ensemble models in this 38-minute Databricks talk. Learn to transform development code into scalable solutions for category-specific predictions, dramatically reducing runtime from hours to minutes. Explore the general usage, types of PandasUDFs, strategies for overcoming data limits, and equivalent approaches in R and Koalas. Gain insights into applying this method to scale from single models to entire ensembles, enhancing prediction accuracy across diverse categories.

Introduction
The Problem
PandasUDFs
Use Cases
Data Limits
Other Frameworks