Achieving vLLM Platform Portability with Triton Autotuning - Ray Summit 2024
Offered By: Anyscale via YouTube
Course Description
Overview
Explore a conference talk from Ray Summit 2024 where Burkhard Ringlein of IBM Research delves into achieving platform portability for vLLM using Triton autotuning. Learn about the challenges of vLLM's reliance on hand-written CUDA kernels and how IBM Research's innovative "dejavu" mechanism for the Triton autotuner eliminates production environment autotuning overhead. Discover early results showing significant kernel speed-ups, improved cross-platform performance, and reduced external dependencies for vLLM. Gain insights into Triton-only vLLM deployments and more portable, future-proof solutions for serving Large Language Models across different hardware platforms.
Syllabus
How IBM Research Achieved vLLM Platform Portability with Triton Autotuning | Ray Summit 2024
Taught by
Anyscale
Related Courses
Finetuning, Serving, and Evaluating Large Language Models in the WildOpen Data Science via YouTube Cloud Native Sustainable LLM Inference in Action
CNCF [Cloud Native Computing Foundation] via YouTube Optimizing Kubernetes Cluster Scaling for Advanced Generative Models
Linux Foundation via YouTube LLaMa for Developers
LinkedIn Learning Scaling Video Ad Classification Across Millions of Classes with GenAI
Databricks via YouTube