Automatic Differentiation for Sparse Tensors

Explore a cutting-edge framework for efficient automatic differentiation of sparse tensors in this 15-minute conference talk presented at ACM SIGPLAN's CTSTA'23. Delve into the challenges posed by irregular sparsity patterns in data-intensive applications and discover how this novel approach overcomes substantial memory and computational overheads. Learn about the key aspects of the proposed framework, including a compilation pipeline that leverages two intermediate DSLs with AD-agnostic domain-specific optimizations and efficient C++ code generation. Gain insights into how this innovative solution outperforms state-of-the-art alternatives across various synthetic and real-world sparse tensor datasets, potentially revolutionizing the field of automatic differentiation for sparse tensor operations.

[CTSTA'23] Automatic Differentiation for Sparse Tensors