Building for Operability: Stateful Applications in Containers with the Autopilot Pattern - Philly ETE 2017

Explore the challenges and solutions of deploying stateful applications in containers through this comprehensive conference talk. Dive into the Autopilot Pattern, a powerful design concept that addresses operational constraints in container frameworks. Learn how to build intelligent architectures that maximize developer flexibility and minimize human intervention by moving lifecycle event handling responsibilities into the application. Discover how to apply the Autopilot Pattern to classic legacy stateful application architectures, covering topics such as replication bootstrapping, automated fail-over, and backups without custom orchestration. Gain insights into solving problems like service discovery, load balancing, and automated fail-over in containerized environments. Explore the concept of immutable deployments, minimal overhead, and the unit of deployment in container ecosystems. Understand the challenges of data persistence in containers and learn about solutions like host volumes and infrastructure-backed discovery bootstrap. Delve into the guarantees and considerations for asynchronous replication, failover scenarios, and consistency between nodes. Empower development teams to operationalize their applications effectively in containerized environments.

Intro
Why stateful containers?
Immutable: deploy a versioned artifact
Minimal overhead: ship only* the application
A container is the unit of deployment.
A group of containers is the unit of deployment (ex. K8s pods, Swarm stack, Triton deployment )
A container can't persist data.
Problem: Service Discovery
Problem: Load Balancing
Problem: Automated Fail-over
Host volumes solve the problem of persistence in containers.
orchestrator and
3 requirements
Ability to provision containers across multiple compute nodes
Network virtualization (ex. VXLan)
IP inside the container
Infrastructure-backed discovery bootstrap
We might need some help
habitat
Know your guarantees: Asynchronous replication Unavailable for writes during failover Possible stale reads during failover . May drop client-acknowledged writes during failover Consistent between nodes after failover
Initialization
Empower development teams to operationalize their applications