Kafka, DevOps, and Resilience for All

Explore DevOps practices and resilience strategies for Apache Kafka in this conference talk from GOTOpia Chicago 2021. Learn how to apply infrastructure as code (IAC) principles to Kafka-based applications, covering topic and partition creation, secrets management, and configuration options. Discover tools and techniques for improving repeatability, resilience, and observability in Kafka deployments. Gain insights into monitoring, backup management, and best practices for resilience. Examine specific examples of how these tools have helped in real-world scenarios and potential pitfalls to avoid. Delve into various implementation options, including Bash scripts, Terraform, and Kubernetes-native solutions, to enhance your Kafka infrastructure management skills.

Intro
Kafka - what's the difference?
Key concepts
What can I DevOpsify?
What are the options?
Bash
Terraform
Kafka config manager
Secrets management
In an ideal world
In a k8s-native world
Backups
Backup options
What about application parameters?
Monitoring
Monitoring tooling
Tooling - commercial
Tooling - open source
Key takeaways
Outro