YoVDO

Rammer - Enabling Holistic Deep Learning Compiler Optimizations with rTasks

Offered By: USENIX via YouTube

Tags

OSDI (Operating Systems Design and Implementation) Courses Deep Learning Courses Heuristics Courses Parallel Computing Courses Compiler Design Courses Scheduling Algorithms Courses

Course Description

Overview

Explore a groundbreaking approach to optimizing Deep Neural Network (DNN) computation on hardware accelerators in this 17-minute conference talk from OSDI '20. Dive into the innovative Rammer compiler design, which generates efficient static spatio-temporal schedules for DNNs at compile time, minimizing scheduling overhead and maximizing hardware utilization. Learn how Rammer employs novel, hardware-neutral abstractions for computation tasks and accelerators, enabling holistic exploitation of parallelism through inter- and intra-operator co-scheduling. Discover the significant performance improvements achieved by Rammer over state-of-the-art compilers like TensorFlow XLA and TVM, as well as vendor-optimized libraries like NVIDIA's TensorRT. Gain insights into Rammer's implementation across multiple hardware backends, including NVIDIA GPUs, AMD GPUs, and Graphcore IPU, and understand its potential impact on efficient DNN execution in massively parallel environments.

Syllabus

OSDI '20 - Rammer: Enabling Holistic Deep Learning Compiler Optimizations with rTasks


Taught by

USENIX

Related Courses

Intro to Parallel Programming
Nvidia via Udacity
Introduction to Linear Models and Matrix Algebra
Harvard University via edX
Введение в параллельное программирование с использованием OpenMP и MPI
Tomsk State University via Coursera
Supercomputing
Partnership for Advanced Computing in Europe via FutureLearn
Fundamentals of Parallelism on Intel Architecture
Intel via Coursera