YoVDO

Multi-Table Transactions with LakeFS and Delta Lake - Tech Talk

Offered By: Databricks via YouTube

Tags

Data Lakes Courses CI/CD Courses Version Control Courses Data Management Courses Data Pipelines Courses Delta Lake Courses LakeFS Courses

Course Description

Overview

Explore multi-table transactions with LakeFS and Delta Lake in this 45-minute tech talk recording from Databricks. Learn how LakeFS enables collaborative data lake management and CI/CD deployment of data, while Delta Lake facilitates building a Lakehouse architecture on various storage systems. Discover the integration of these technologies to simplify multi-table pipelines. Gain insights from speakers Paul Singman of Treeverse and Denny Lee of Databricks as they discuss typical data lake projects, challenges, and solutions. Follow along with demonstrations of the LakeFS UI, repository setup, DataFrame operations, and merging techniques. Understand concepts like single table consistency, validation notebooks, Git integration, and concurrent operations. Enhance your knowledge of modern data lake management and Lakehouse architectures through this informative presentation.

Syllabus

Intro
Presentation
Typical Data Lake Project
Data Lake Problems
Single Table Consistency
MultiTable Transactions
Demo
LakeFS UI
Demo Overview
Demo Repository
DataFrame
Second Table
Commit
Merge
Merge Failed
Retry Merge
Validation Notebook
Questions
Git Integration
Concurrent Operations
Summary


Taught by

Databricks

Related Courses

Données et services numériques, dans le nuage et ailleurs
Certificat informatique et internet via France Université Numerique
Introduction to Digital Curation
University College London via Independent
Excel Avanzado
Miríadax
SAP Business Warehouse powered by SAP HANA
SAP Learning
Programming Mobile Applications for Android Handheld Systems: Part 2
University of Maryland, College Park via Coursera