Streamlining Big Data Workflows with Memoization and Work Avoidance

Discover how to optimize big data and machine learning workflows using Argo Workflows in this informative conference talk. Learn about the power of memoization and work avoidance techniques to significantly reduce runtime and compute costs. Explore the inner workings of memoization in Argo, making it accessible for new users to implement. Gain insights into various approaches for work avoidance and best practices derived from real-world community experiences. By the end of the presentation, acquire the knowledge to leverage memoization and work avoidance strategies in your own workflows, leading to substantial improvements in efficiency. Additionally, get a glimpse into the future roadmap for enhancing these features based on valuable community feedback.

Streamlining Big Data Workflows with Memoization & Work Avoidance - Alan Clucas & Julie Vogelman