Speeding Up Analysis Pipelines with Remote Container Images

Explore how CERN optimizes analysis pipelines using remote container images in this 30-minute conference talk from KubeCon + CloudNativeCon Europe 2021. Discover the containerization process for massive software releases and learn about the distribution of container images across a global network of sites. Gain insights into lazy loading techniques using containerd remote snapshotter, which ensures rapid startup times and reduces network traffic. Witness a live demonstration of a real physics analysis pipeline featuring hundreds of parallel jobs. Delve into topics such as CERN's infrastructure, container efficiency, lazy pulling experiments, software distribution, deployment workflows, and potential improvements in this informative presentation by Ricardo Rocha and Spyridon Trigazis from CERN.

Introduction
What is CERN
CERN VMFS
Container Efficiency
History
Lazy Pulling
Experiments
Demonstration
Software Distribution
Deployment Overview
Workflow Overview
Network Traffic
Snapshotter Status
Improvements
Closing remarks