A Better and More Efficient ML Experience for CERN Users

Explore how CERN leverages machine learning and Kubeflow to handle massive data growth from the Large Hadron Collider in this 24-minute conference talk. Discover the challenges posed by petabytes of annual data and learn about the innovative solutions implemented to improve resource usage, manage credentials, and ensure security. Gain insights into the integration of Kubeflow with site services, the use of tools like Harbor, Trivy, OPA, and Falco for reproducible and secure workflows, and the impact of centralized resources on overall efficiency. Understand how these advancements are crucial for processing the anticipated 10x increase in data from upcoming LHC upgrades and supporting thousands of physicists worldwide in their analysis efforts.

Intro
Large Hadron Collider - LHC
ML in LHC Data Acquisition
ML to Discover New Physics
Motivation for Kubeflow
Kubeflow at CERN
Improved Resource Usage
Resources
Integration
Credential Management
Namespace Management
Scans and Runtime Checks
User Feedback
Conclusions
Questions?