DataFusion and Apache Arrow: Supercharging Data Analytics with a Rust-Based Query Engine
Offered By: Databricks via YouTube
Course Description
Overview
Discover how Rust, Apache Arrow, and the Data Fusion Query Engine are revolutionizing modern data stacks in this 29-minute video from Databricks. Explore the implementation timeline for new database systems, LLVM-like infrastructure for databases, and DataFusion's growth and milestones. Learn about DataFusion's features, including SQL support and extensibility, and examine real-world applications like VegaFusion and Cube.js. Gain insights into future directions for distributed computing with Ballista and understand how these technologies can supercharge your data analytical tools.
Syllabus
Introduction
What is going on?
Apache Arrow
Implementation timeline for a new Database system
LLVM Compiler Infrastructure
LLVM-like Infrastructure for Databases
DataFusion Project Growth
DataFusion Milestones: Time to Mature
Initial Logical Plan
Let's Optimize!
The Initial Execution Plan
DataFusion Features
SQL Support
Extensibility
VegaFusion
Cube.js / Cube Store
Coralogix
blaze-rs
Ballista Distributed Compute
Future Directions
Taught by
Databricks
Related Courses
Cloud Computing Concepts, Part 1University of Illinois at Urbana-Champaign via Coursera Cloud Computing Concepts: Part 2
University of Illinois at Urbana-Champaign via Coursera Reliable Distributed Algorithms - Part 1
KTH Royal Institute of Technology via edX Introduction to Apache Spark and AWS
University of London International Programmes via Coursera Réalisez des calculs distribués sur des données massives
CentraleSupélec via OpenClassrooms