Achieving vLLM Platform Portability with Triton Autotuning - Ray Summit 2024
Offered By: Anyscale via YouTube
Course Description
Overview
Explore a conference talk from Ray Summit 2024 where Burkhard Ringlein of IBM Research delves into achieving platform portability for vLLM using Triton autotuning. Learn about the challenges of vLLM's reliance on hand-written CUDA kernels and how IBM Research's innovative "dejavu" mechanism for the Triton autotuner eliminates production environment autotuning overhead. Discover early results showing significant kernel speed-ups, improved cross-platform performance, and reduced external dependencies for vLLM. Gain insights into Triton-only vLLM deployments and more portable, future-proof solutions for serving Large Language Models across different hardware platforms.
Syllabus
How IBM Research Achieved vLLM Platform Portability with Triton Autotuning | Ray Summit 2024
Taught by
Anyscale
Related Courses
High Performance ComputingGeorgia Institute of Technology via Udacity Fundamentals of Accelerated Computing with CUDA C/C++
Nvidia via Independent High Performance Computing for Scientists and Engineers
Indian Institute of Technology, Kharagpur via Swayam CUDA programming Masterclass with C++
Udemy Neural Network Programming - Deep Learning with PyTorch
YouTube