Operating Enterprise Grade Kubernetes Clusters at Salesforce on Bare Metal

Explore the challenges and solutions of operating enterprise-grade Kubernetes clusters on bare metal infrastructure in this 32-minute conference talk by Anubhav Dhoot and Mayank Kumar from Salesforce. Gain insights into managing highly available Kubernetes clusters without public cloud managed offerings or VMs, focusing on specific challenges in setup, security, upgrades, and 24/7 monitoring using Puppet. Learn from the experiences of managing a fleet of 2600+ hosts across 20+ data centers over four years, including battle stories on etcd configuration issues, hairpin networking problems, and service account key rotation. Discover the intricacies of Salesforce's on-premise Kubernetes platform, delve into Puppet module features, and understand the operational aspects of monitoring bare metal Kubernetes. Explore potential pitfalls such as mounting hostPath, bridge networking failures, and inconsistent API server flags. This talk serves as a valuable resource for teams managing their own Kubernetes clusters, offering lessons learned and cautionary tales from Salesforce's journey towards managed Kubernetes offerings in the public cloud.

Introduction
Overview of Salesforce on premise Kubernetes platform
Puppet module deep dive
Puppet module features
Kubernetes on Puppet review
Operationalize
Monitoring bare metal Kubernetes
Perils of mounting hostPath
Bridge networking failures
Bullying quorum members
Handle sharp tools with care
Inconsistent Api Servers flags
Summary