Managing Artifacts at Scale for CI and Data Processing

Explore best practices for managing artifacts at scale in CI and data processing workflows using Argo Workflows. Learn how to configure artifact repositories with Key-only Artifacts, implement Artifact Garbage Collection strategies, and optimize workflow management for large-scale operations. Discover techniques to simplify artifact handling, automate cleanup processes, and scale up Argo Workflows instances effectively. Through practical examples covering both CI and data use cases, gain insights into advanced artifact management approaches that can be applied to your own workflows, regardless of their complexity or scale.

Managing Artifacts at Scale for CI and Data Processing - Caelan Urquhart & Julie Vogelman