Beyond Malloc Efficiency to Fleet Efficiency - A Hugepage-Aware Memory Allocator

Explore a 14-minute conference talk from OSDI '21 that delves into TEMERAIRE, a hugepage-aware enhancement of TCMALLOC designed to optimize memory allocation at warehouse scale. Learn how this innovative approach goes beyond traditional malloc efficiency to improve fleet-wide productivity by maximizing hugepage coverage and minimizing fragmentation overheads. Discover the design and implementation strategies behind TEMERAIRE, including its impact on reducing CPU overheads in application code. Examine the results of application studies across 8 different applications, showcasing improvements in requests-per-second and RAM usage. Gain insights from a large-scale experiment and longitudinal rollout in Google's warehouse scale computers, revealing significant reductions in TLB miss stalls and memory fragmentation. Conclude with a discussion on enhancing allocator development processes and potential optimization strategies for future memory allocators.

Intro
normally reducing malloc cycles is good
business logic changes, infrastructure does not
malloc cycles do not matter. $$$ spent on hardware matter.
most cpu cycles do nothing
hugepages cheapen the page table walk
hugepages make the TLB bigger!
space efficient hugepage aware allocators are hard
demand oscillates wildly
emptying density - binpacking
mistakes can live forever
tcmalloc structure
spans back everything
change nothing but the page heap
Temeraire: the design
slack and donation
how does the HugeFiller make decisions?
HugeFiller tracks metadata per hugepage
we favor fragmentation over fullness
results
staged rollout
saved -1.3% of cycles
saving memory in the process
virtuous cycles: hugepage coverage