Approximate Caching for Efficiently Serving Text-to-Image Diffusion Models

Explore a conference talk presenting NIRVANA, an innovative system for text-to-image generation using diffusion models. Discover how approximate caching techniques can significantly reduce computational requirements and latency in producing high-quality images from text prompts. Learn about the novel cache management policy that enables 21% GPU compute savings, 19.8% end-to-end latency reduction, and 19% cost savings in real production environments. Gain insights into the extensive characterization of production text-to-image prompts, focusing on caching, popularity, and reuse of intermediate states in large-scale deployments. Understand the potential impact of this research on making resource-intensive diffusion models more efficient and accessible for various applications.

NSDI '24 - Approximate Caching for Efficiently Serving Text-to-Image Diffusion Models