GPU Programming in Julia - Workshop

Dive into GPU programming with Julia in this comprehensive 3-hour workshop from JuliaCon 2021. Explore three major GPU programming packages: CUDA.jl for NVIDIA GPUs, AMDGPU.jl for AMD GPUs, and oneAPI.jl for Intel GPUs. Learn various approaches for GPU programming, from user-friendly generic array operations to performance-focused hardware-specific kernels. Cover topics including GPU architecture, parallel programming models, array and kernel programming, profiling techniques, and common issues. Engage with practical case studies on image processing, machine learning, and random number generation. Gain hands-on experience solving real-world problems like the heat equation using GPUs. Conclude with a sneak peek at the Enzyme automatic differentiation framework and a Q&A session on future developments in Julia GPU programming.

Welcome!.
Welcome.
Outline.
JuliaGPU packages.
JuliaGPU back-ends.
GPU Architecture.
Parallel programming models.
Follow along and links to notebooks, JuliaHub.
Start of tuturial with notebook.
Array programming.
Kernel programming.
Parallel programming + questions.
Profiling.
Profiling: NVIDIA Nsight Systems: live example.
Profiling: NVIDIA Nsight Compute: live example → optimize single kernel invocation.
Common issues: unsupported array operations.
Common issues: unsuppored kernel operations.
Parallel programming issues.
Tour of accompanying Github repo.
Case Study I: Image processing using AMDGPU.
Break.
Case Study II: Fun with arrays, Machine Learning.
Case Study III: Random number generators.
Kernel abstractions.
Example: Solving heat equation with GPU.
Sneak peek of Enzyme (automatic differentiation framework).
Questions and Future plans.