CloudCluster - Unearthing the Functional Structure of a Cloud Service

Explore the functional structure of cloud services through a 15-minute conference talk from NSDI '22. Delve into CloudCluster, an innovative algorithm that uses VM-to-VM traffic matrices to uncover the underlying architecture of cloud services. Learn how this approach overcomes challenges of scale and measurement noise, achieving over 92% clustering accuracy. Discover practical applications for cost reduction, anomaly detection, and misconfiguration identification in cloud environments. Gain insights into cloud service anatomy, monitoring abstractions, and the importance of functional structure visibility for both providers and customers.

Intro
Anatomy of a Cloud Service: The Customer View
Anatomy of a Cloud Service: The Provider View
Today: Monitoring at Different levels of Abstract
A New Abstraction: The Functional Structure
What do we mean by Functional Structure?
The Provider Does Not Know the Functional Stru
Extracting the Functional Structure
Hypothesis
Approach: Use Clustering!
Cloud Cluster: Challenges
The Problem
Insight: Exploit Redundancy in VM-to-VM Traffic
Dimensionality Reduction
Agglomerative Clustering
Generating the Clusters: Where to cut the dendre
Use Cluster Inconsistency to Decide!
Evaluating Clustering Performance
Cloud Cluster. Excellent Agreement with Ground
What is the Functional Structure Useful For?
Visualization
Detecting Traffic Shifts with Cloud Cluster
Detecting Structural Changes
Anomaly Detection
Summary