THC - Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression

Explore a groundbreaking conference talk on Tensor Homomorphic Compression (THC), a novel bi-directional compression framework designed to accelerate distributed deep learning. Delve into the challenges of communication overhead in large-scale neural network training and discover how THC addresses these issues by enabling direct aggregation of compressed values. Learn about the framework's compatibility with in-network aggregation (INA) and its significant performance improvements over state-of-the-art systems. Gain insights into THC's ability to reduce computational overheads at the parameter server, minimize compression error, and ultimately achieve faster training times and higher accuracy for representative vision and language models.

NSDI '24 - THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression