A Hybrid Container Cloud With Kubernetes and Hadoop YARN

Explore a framework developed by Alibaba that enables seamless integration of Kubernetes and Hadoop YARN in a single cluster, allowing for elastic resource sharing between different types of workloads. Discover the challenges and solutions in managing both long-running services and big data applications within a unified environment. Learn about the architectural differences between Kubernetes and Hadoop YARN, and understand the motivations behind creating a hybrid container cloud. Gain insights into the requirements, hurdles, and lessons learned from implementing this approach in Alibaba's massive commercial platform. Examine topics such as data center utilization, workload characteristics, resource sharing techniques, CPU cgroups, time-sliced scheduling, and API design for this hybrid system. Understand how this framework addresses the operational and hardware costs associated with maintaining separate clusters while supporting diverse workloads efficiently.

Intro
Data center AVG utilization
Workloads in Alibaba data center
Before: service & Job separated cluster
Services vs Jobs
Google Borg
What is co-location
resource sharing
CPU ogroup
Together time sliced scheduling
Normal day Sales day
Design Principal
API Design
Scheduling Unit
Communication
Container Runtime
Larger scale