HammingMesh: A Network Topology for Large-Scale Deep Learning
Offered By: Scalable Parallel Computing Lab, SPCL @ ETH Zurich via YouTube
Course Description
Overview
Explore a groundbreaking network topology for large-scale deep learning systems in this award-winning conference talk from ACM/IEEE Supercomputing 2022. Delve into the HammingMesh design, a novel approach developed by the Scalable Parallel Computing Lab at ETH Zurich to address data movement challenges in AI training. Discover how HammingMesh provides high bandwidth at low cost with enhanced job scheduling flexibility, supporting full bandwidth and isolation for deep learning training jobs with two-dimensional parallelism. Learn about its capacity to handle high global bandwidth for generic traffic, positioning it as a crucial component for future AI systems with extreme bandwidth requirements. Gain insights into the workload analysis that informed the topology's design and understand how HammingMesh aims to overcome the performance limitations of current training systems, potentially unlocking the next phase of growth in modern AI.
Syllabus
HammingMesh: A Network Topology for Large-Scale Deep Learning
Taught by
Scalable Parallel Computing Lab, SPCL @ ETH Zurich
Related Courses
Financial Sustainability: The Numbers side of Social Enterprise+Acumen via NovoEd Cloud Computing Concepts: Part 2
University of Illinois at Urbana-Champaign via Coursera Developing Repeatable ModelsĀ® to Scale Your Impact
+Acumen via Independent Managing Microsoft Windows Server Active Directory Domain Services
Microsoft via edX Introduction aux conteneurs
Microsoft Virtual Academy via OpenClassrooms