DeepSketch - A New Machine Learning-Based Reference Search Technique for Post-Deduplication Delta Compression

Explore a cutting-edge machine learning-based reference search technique for post-deduplication delta compression in this 15-minute conference talk from FAST '22. Learn about DeepSketch, a novel approach that leverages deep neural networks to extract data block sketches, improving data-reduction efficiency in storage systems. Discover how this technique outperforms existing methods, achieving up to 33% better data-reduction ratios across eleven real-world workloads. Gain insights into the challenges of reference search in delta compression and how DeepSketch addresses them through innovative data clustering and training data set processing. Understand the methodology used to evaluate DeepSketch's performance and its overall benefits in minimizing data center management costs.

Intro
Executive Summary
Data Reduction in Storage Systems
Post-deduplication Delta Compression Combines three different data-reduction approaches
Overview of Post-Deduplication Delta Compression
Lossless Compression
Key Challenge: Reference Search How to find a good reference block for an incoming data block across a wide range of stored data at low cost
Limitations of Existing Techniques - Provide significantly lower data-reduction ratios than the optimal
DeepSketch: Key Idea Use the learning-to-hash method for sketch generation A promising machine learning (ML).-based approach for the
DeepSketch: Challenges Lack of semantic information
Data Clustering for DeepSketch . Existing clustering algorithms are unsuitable for DeepSketch
Post-Processing for Training Data Set Non-uniform distribution of data blocks across the clusters
Evaluation Methodology Compared data-reduction techniques
Overall Data-Reduction Benefits
Performance Overhead