Developing Data Pipelines with Branch Deployments - A New Approach
Offered By: Databricks via YouTube
Course Description
Overview
Explore a 15-minute conference talk that challenges traditional development patterns for data pipelines. Learn how Enigma leveraged Databricks and Dagster's branch deployments to create a highly productive workflow for developing data pipelines on production data safely. Discover how developers can work on feature branches, push to branch deployments, and utilize cloud infrastructure and branched storage for improved productivity, more trusted data, and lower costs. Gain insights from Ryan Green, CTO of Enigma, on replicating the fast testing and tight feedback loops of traditional software engineering in data engineering. Access additional resources on data engineering and the Databricks Lakehouse Platform, and connect with Databricks through various social media platforms for more information.
Syllabus
Sponsored By: Dagster Labs | Dev/Stage/Prod is the Wrong Pattern for Data Pipelines
Taught by
Databricks
Related Courses
内存数据库管理openHPI CS115x: Advanced Apache Spark for Data Science and Data Engineering
University of California, Berkeley via edX Processing Big Data with Azure Data Lake Analytics
Microsoft via edX Google Cloud Big Data and Machine Learning Fundamentals en Español
Google Cloud via Coursera Google Cloud Big Data and Machine Learning Fundamentals 日本語版
Google Cloud via Coursera