Fast Erasure Coding for Data Storage - A Comprehensive Study of the Acceleration Techniques

Explore a comprehensive study on accelerating erasure coding techniques for data storage in this USENIX FAST '19 conference talk. Delve into various optimization methods including bitmatrix design, computation scheduling, XOR operation reduction, caching management, and vectorization. Learn how these techniques, previously proposed individually, can be combined for maximum efficiency. Discover a simple design approach that optimizes bitmatrices using a weighted cost function, and understand a straightforward erasure coding procedure that outperforms many existing techniques. Compare the performance against well-known codes like EVENODD, RDP, and STAR. Gain insights into why vectorizing XOR operations may be preferable to vectorizing finite field operations, considering both encoding throughput and CPU migration efforts. Examine individual acceleration techniques, optimization tiers, and their combinations through extensive testing results.

Intro
Data Reliability
Replication - Erasure Code
Acceleration Techniques
Cauchy Reed-Solomon Codes
Fast GF Ops - Binary Representation
Techniques Tiers
Bitmatrix Normalization (BN)
Smart Scheduling (SS)
Scheduling - Cache Optimization (S-CO)
Vectorization
Question to Answer
Individual Techniques
Optimization Tiers
Combinations (ij)-strategy
Choice of Cauchy Matrix
Cost Function Improvement
Proposed Coding Procedure
Testing Setup
Encoding v.s. Efficient RS/CRS code
Encoding v.s. Three Parities Codes
Encoding v.s. Two Parities Codes
Overall Encoding Improvement
Decoding Throughput
Conclusion