Mastering the Maze: Navigating Complexity in Distributed Systems

Dive into a comprehensive conference talk that unravels the complexities of distributed systems. Explore the challenges and strategies for navigating microservices architecture, including unreliable networks, concurrency issues, and consistency types. Learn about crucial concepts like timeout mechanisms, retry strategies, idempotency, and circuit breakers. Discover solutions for dual write problems, process pauses, and observability concerns. Delve into cybernetics principles, systems thinking, and SRE best practices. Gain insights on infrastructure as code, chaos engineering, and measuring complexity in distributed environments. Master the intricacies of distributed system design and implementation to build more resilient and scalable applications.

intro
preamble
agenda
what is a distributed system?
what is complexity?
monolithic architecture
disadvantages
microservices architecture
what do distributed systems give us?
challenges, quality attributes
what are main troubles?
unreliable networks
strategy: timeout
strategy: retry
strategy: idempotency
strategy: circuit breaker
concurrency and lost writes
strategy: snapshot isolation
strategy: compare and set
strategy: lease
dual write problem
strategy: transactional outbox
strategy: log tailing
unreliable clocks
availability and consistency
high availability
failure
consistency types
linearizability
strategy: distributed consensus algorithm, e.g. raft
more complexities
eventual consistency
strategy: read from leader
process pauses
strategy: fencing
observability
strategy: distributed tracing
strategy: orchestration over choreography
evolvability and cybernetics principles
systems thinking
feedback loops
adaptability and learning
goal-oriented design
big ball of mud
hierarchy
fallacy: all microservices are the same
strategy: service types
sre principles
infrastructure as code
chaos engineering and testing: jepsen tests
simplicity and measuring complexity
thank you for attending!