A Tensor Compiler Approach for One-size-fits-all ML Prediction Serving

Explore a groundbreaking approach to machine learning model scoring in this 20-minute conference talk from OSDI '20. Discover Hummingbird, an innovative tensor compiler method that simplifies and optimizes ML prediction serving for enterprise applications. Learn how this technique compiles featurization operators and traditional ML models into a compact set of tensor operations, reducing infrastructure complexity and leveraging existing Neural Network compilers and runtimes. Examine the performance benefits of Hummingbird, which competes with and often surpasses hand-crafted kernels on both CPU and GPU micro-benchmarks while enabling seamless end-to-end acceleration of ML pipelines. Gain insights into how this open-source solution addresses the challenges of ML adoption in enterprise environments by streamlining the model scoring process and enhancing efficiency across various hardware platforms.

OSDI '20 - A Tensor Compiler Approach for One-size-fits-all ML Prediction Serving