Snowcannon - Synchronous Data Replication in FDB

Explore the design and implementation of Snowcannon, a synchronous data replication system for FoundationDB (FDB), in this conference talk. Learn how Snowflake addresses the limitations of FDB's current asynchronous disaster recovery solution by developing a distributed queueing system that enables synchronous transaction replication. Discover the benefits of Snowcannon, including its ability to replay rollbacks and redos, support maintenance of the secondary cluster without primary downtime, and ensure zero data loss recovery from backups. Delve into the replication protocol, failover switch, queueing architecture, and quorum protocol that form the core of Snowcannon's design. Gain insights into the challenges of building a distributed system, including the addition of new machine classes and roles. Understand how Snowcannon meets Snowflake's metadata requirements and compare it to other replication solutions.

Intro
OUTLINE
SNOWFLAKE REQUIREMENTS
COMPARISON OF REPLICATION SOLUTIONS
HOW TO BUILD A DISTRIBUTED SYSTEM
WE ADD A NEW MACHINE CLASS
WE ADD SUPPORT TO RECRUIT THE NEW ROLE
ADD SUPPORT TO THE WORKER
CONCLUSIONS
HIGH LEVEL VIEW OF ARCHITECTURE
SNOWFLAKE DATA REPLICATION
SNOWCANNON SWITCH
CHALLENGES WITH PRODUCER RECOVERY
SnowCannon, designed for Snowflake Metadata Requirements