Efficient CHAD - Optimizing Combinatory Homomorphic Automatic Differentiation

Explore an optimized approach to Combinatory Homomorphic Automatic Differentiation (CHAD) in this 19-minute conference talk from POPL 2024. Discover how researchers from Utrecht University have enhanced the basic CHAD algorithm using well-known methods to create a simple, composable, and widely applicable reverse-mode automatic differentiation technique. Learn about the implementation of sparse vectors, state-passing style code, and functional mutable updates to achieve the correct computational complexity expected in reverse-mode AD. Examine the Agda formalization of the complexity proof and understand how these techniques can be applied to differentiate parallel functional array programs. Gain insights into preserving task-parallelism and writing data-parallel derivatives for array primitives. Access the accompanying article and supplementary archive for a deeper dive into this research on efficient automatic differentiation in functional programming.

[POPL'24] Efficient CHAD