ModelKeeper - Accelerating DNN Training via Automated Training Warmup

Explore a 14-minute conference talk from USENIX NSDI '23 that introduces ModelKeeper, an innovative system for accelerating deep neural network (DNN) training. Learn how this automated training warmup approach repurposes previously-trained models in shared clusters to jump-start new training jobs. Discover the key insights behind ModelKeeper, including its ability to identify architectural similarities between models, select suitable parent models, and perform structure-aware weight transformations. Understand how this system can significantly reduce training time across various computer vision and natural language processing models without compromising accuracy. Gain valuable insights into improving efficiency in machine learning workflows, particularly in environments with multiple neural architecture search and training jobs.

NSDI '23 - ModelKeeper: Accelerating DNN Training via Automated Training Warmup