Efficient Data Parallel Distributed Training with Flyte, Spark and Horovod
Offered By: Linux Foundation via YouTube
Course Description
Overview
Explore efficient data parallel distributed training techniques using Flyte, Spark, and Horovod in this 41-minute conference talk presented by Ketan Umare and Katrina Rogan from Union.ai. Gain insights into the integration of these powerful tools for optimizing machine learning workflows. Learn about Flyte's architecture, concepts, and user journey, including workflow creation, registration, and execution. Discover how to leverage Spark for data processing and Horovod for distributed deep learning. The presentation covers key topics such as averages, code examples, stack traces, and an example scenario, providing a comprehensive overview of the subject matter. Enhance your understanding of distributed training methodologies and their practical applications in modern data science and machine learning projects.
Syllabus
Introduction
Agenda
Recap
Overview
Averages
Spark
What is Flyte
Workflows
User Journey
Code Example
Registration
Launching an execution
Graph of execution
Stack trace
Flyte concepts
Flyte architecture
Demo
Example Scenario
Taught by
Linux Foundation
Tags
Related Courses
Observability in the MLOps Lifecycle with PrometheusUSENIX via YouTube Building Robust ML Production Systems Using OSS Tools for Continuous Delivery for ML
Linux Foundation via YouTube Self-serve Feature Engineering Platform Using Flyte and Feast
Linux Foundation via YouTube Enforcing Data Quality in Data Processing and ML Pipelines with Flyte and Pandera
Linux Foundation via YouTube Flyte - Cloud Native Machine Learning and Data Processing Platform
Linux Foundation via YouTube