Neon: Serverless PostgreSQL - Separation of Storage and Compute

Explore the innovative concept of serverless PostgreSQL in this comprehensive talk by Heikki Linnakangas from Neon. Delve into the architecture of Neon, understanding its separation of storage and compute, and how it leverages PostgreSQL's Write-Ahead Log. Learn about the write and read paths through Pageservers, the control plane and proxy implementation, and the unique storage engine design. Discover the advantages of this approach over traditional Point-In-Time Recovery and existing LSM implementations. Gain insights into Neon's storage format, search capabilities, image layer creation, compaction, and garbage collection processes. Explore the branching feature and engage with open questions surrounding this cutting-edge database technology. This seminar, part of the CMU Database Group's "¡Databases!" series, offers a deep dive into the future of cloud-native database solutions.

Intro
What is Neon?
We love PostgreSQL
Separation of Storage and Compute
Why separate compute and storage?
Detour: PostgreSQL Write-Ahead Log
Write path: Pageservers
Read path: Pageservers
Control plane and proxy
Traditional Point-In-Time Recovery
Storage engine
Why not use an existing LSM implementation?
Storage format
Search
Image layer creation
Compaction
Garbage collection
Branching
Open questions