Accelerating Graph Neural Networks with Fine-Grained Intra-Kernel Communication-Computation Pipelining on Multi-GPU Platforms

Explore a cutting-edge conference talk on accelerating Graph Neural Networks (GNNs) using multi-GPU platforms. Dive into the innovative MGG system design, which introduces a novel fine-grained dynamic software pipeline for optimizing computation-communication overlapping within GPU kernels. Learn about GNN-tailored pipeline construction, GPU-aware pipeline mapping, and intelligent runtime design with analytical modeling and optimization heuristics. Discover how MGG outperforms state-of-the-art full-graph GNN systems, achieving significant speed improvements over DGL, MGG-UVM, and ROC frameworks. Gain insights into addressing the challenges of processing large input graphs for GNNs and the importance of joint scheduling and optimization of computation and communication operations for high-performance delivery.

OSDI '23 - MGG: Accelerating Graph Neural Networks with Fine-Grained Intra-Kernel Communication...