Mesos - A Platform for Fine-Grained Resource Sharing in the Data Center

Explore a 30-minute conference talk from USENIX NSDI '11 that introduces Mesos, a platform designed for fine-grained resource sharing in data centers. Discover how Mesos enables efficient sharing of commodity clusters among multiple diverse computing frameworks like Hadoop and MPI. Learn about the platform's ability to improve cluster utilization, avoid data replication, and achieve data locality through a unique turn-based approach. Understand the innovative distributed two-level scheduling mechanism called resource offers, which allows Mesos to allocate resources while frameworks decide on their usage. Gain insights into Mesos' scalability, tested up to 50,000 emulated nodes, and its resilience to failures. Delve into the talk's comprehensive coverage of Mesos' goals, design elements, architecture, optimization techniques, and comparisons with static partitioning. Conclude with an exploration of related work and a discussion on sharing datacenter networks.

Intro
Background
Problem
Where We Want to Go
Solution
Other Benefits of Mesos
Outline
Mesos Goals
Design Elements
Element 1: Fine-Grained Sharing
Element 2: Resource Offers
Mesos Architecture
Optimization: Filters
Dynamic Resource Sharing
Mesos vs Static Partitioning
Fault Tolerance
Related Work
Conclusion
Sharing the Datacenter Network