It's All a Numbers Game - The Dirty Little Secret of Scalable Systems

Explore the intricacies of scalable systems in this 48-minute conference talk from GOTO Aarhus 2012. Delve into the core concepts of system scaling, cost-effectiveness, and performance optimization with High-Performance Computing Specialist Martin Thompson. Learn about cache-oblivious algorithms, battling contention, and the Disruptor pattern. Discover strategies for connection multiplexing, capacity planning, and efficient logging practices. Examine the importance of separating read and write operations, utilizing SSDs and Memcache, and embracing simplicity in system design. Gain insights into mechanical sympathy, structural engineering principles applied to software, and the significance of load testing. Understand the economics of performance and the value of collaborative teamwork in building scalable systems.

Intro
Scaling
Costs
Core model
Performance test
Cache oblivious algorithms
Battling contention
Disruptor
Connection multiplexing
Single points of contention
Queues
Coffee shop analogy
Capacity planning
Case logging
We need to log
Separating reading and writing
Updates and replacements
SSDs
Memcache
Be simple
Perfect is not the answer
Mechanical sympathy
Not good enough
Structural engineering
Load testing
Testing to breaking point
When the replica takes over
When theres no load
Avoid being commercial
Eat together
Work together
Performance
Economics
Logging Framework