Building a 5-Exaflop Supercomputer for Meta-AI Research and Large-Scale Model Training
Offered By: USENIX via YouTube
Course Description
Overview
Explore the architecture, construction, and operation of Meta's cutting-edge AI Research SuperCluster in this 35-minute conference talk from SREcon23 Europe/Middle East/Africa. Discover how a small, geographically distributed team of Software and Production Engineers (SRE) collaborated to build and manage one of the world's largest AI supercomputers, boasting 16,000 GPUs and 5 exaflops of compute power. Gain insights into the challenges and solutions involved in supporting large-scale model training, including the recently released Llama series, with a focus on the InfiniBand interconnect, high-performance storage systems, and emerging monitoring and observability needs. Learn valuable lessons from Meta's experience in pushing the boundaries of AI research infrastructure and team collaboration.
Syllabus
SREcon23 Europe/Middle East/Africa - Building a 5-Exaflop Supercomputer for Meta-AI Research and...
Taught by
USENIX
Related Courses
Tech Recruiting Foundations: 2 Understanding Hardware and Operating SystemsLinkedIn Learning Linux Terminal for beginners
Udemy How Do I Compute the Body Electric?
TED via YouTube Analog Supercomputers - From Quantum Atom to Living Body
TEDx via YouTube Self-Driving from Vision Only - Tesla's Self-Driving Progress by Andrej Karpathy - Talk Analysis
Yannic Kilcher via YouTube