The Storage Architecture of Intel's Data Management Platform

Explore the storage architecture of Intel's Data Management Platform (DMP) in this 27-minute conference talk from USENIX Vault '19. Delve into the rack-centric cluster design utilizing an Ethernet-based fabric interconnect and a unique approach to storage redundancy. Learn about the DMP's overview, including its Kubernetes management and use of Container Network and Storage Interface providers. Discover the two volume types: NVMe-over-Fabric based Remote Logical Volumes and Local Logical Volumes optimized for Optane DC Persistent Memory. Gain insights into workload priorities for initial DMP deployments, including a Spark-based AI/Analytics Pipeline and MySQL/MariaDB transactional database on shared storage. Conclude with an update on the DMP effort, preliminary performance results, and future steps for this innovative storage architecture.

Introduction
What is the DMP
What is De disaggregation
Rack Scale Architecture
Capacity Gaps
The Stack
Software Stack
Small Organization
Linstore
Linstore Explanation
Linstore Roadmap
Network Virtualization
Performance Snapshot