SingularFS - A Billion-Scale Distributed File System Using a Single Metadata Server

Explore a groundbreaking conference talk from USENIX ATC '23 that introduces SingularFS, a billion-scale distributed file system utilizing a single metadata server. Delve into the unique challenges faced in fully exploiting the performance of large-scale file systems, including crash consistency overhead, lock contention in shared directories, and NUMA locality. Discover three key techniques implemented in SingularFS: log-free metadata operations, hierarchical concurrency control, and hybrid inode partition. Learn how these innovations eliminate additional crash consistency overheads, maximize metadata operation parallelism, and reduce inter-NUMA access and intra-NUMA lock contention. Gain insights from extensive evaluations demonstrating SingularFS's consistently high performance for metadata operations on both private and shared directories, as well as its steady high throughput for billion-scale directory trees. This 17-minute presentation by researchers from Tsinghua University offers valuable knowledge for those interested in advanced distributed file systems and datacenter technologies.

USENIX ATC '23 - SingularFS: A Billion-Scale Distributed File System Using a Single Metadata Server