Rust Data Engineering

Rust for Data Engineering: Efficient, Safe, and Concurrent Data Processing

• Learn to build robust data processing systems using Rust
• Explore Rust's performance, safety, and concurrency for data tasks

This 4-week course dives deep into leveraging Rust for efficient and reliable data engineering workflows:

• Mastering Rust data structures and collections for data processing
• Leveraging Rust's safety/security features in data engineering context
• Using Rust libraries like Diesel, async, Polars, Apache Arrow
• Interfacing with data stores, REST/gRPC APIs, AWS SDK
• Designing full-fledged data pipelines and processing systems in Rust
• Hands-on projects for building data ingestion tools, ETL pipelines
• Best practices for handling large datasets, optimizing performance
• Techniques for writing safe, concurrent, and lock-free code
• Deploying and maintaining Rust-based data engineering solutions

By the end, you'll gain practical experience building high-performance, secure data systems using Rust - preparing you for real-world data challenges.

\- Module 1: Rust Data Structures: Collections (18 hours)

\- 24 videos (Total 111 minutes)

\- 29 readings (Total 290 minutes)

\- 1 quiz: Rust Collections (Quiz, 30 minutes)

\- 2 assignments

\- Quiz-Getting Started With The Modern Rust Development Ecosystem (Assignment, 180 minutes)

\- Quiz-Rust Sequences and Maps (Assignment, 180 minutes)

\- 5 ungraded labs (60 minutes each)

\- Module 2: Safety, Security and Concurrency with Rust (17 hours)

\- 22 videos (Total 93 minutes)

\- 21 readings (Total 210 minutes)

\- 1 quiz: Safety, Security and Concurrency with Rust (Quiz, 30 minutes)

\- 3 assignments

\- Quiz-Rust Safety and Security Features (Assignment, 180 minutes)

\- Quiz-Security Programming with Rust (Assignment, 180 minutes)

\- Quiz-Rust Concurrency (Assignment, 180 minutes)

\- 3 ungraded labs (60 minutes each)

\- Module 3: Rust Data Engineering Libraries and Tools (13 hours)

\- 21 videos (Total 127 minutes)

\- 14 readings (Total 140 minutes)

\- 1 quiz: Quiz3: Rust Data Engineering Libraries and Tools (Quiz, 30 minutes)

\- 2 assignments

\- Quiz-Using Rust to Manage Data, Files and Network Storage (Assignment, 180 minutes)

\- Quiz-DataFrames with Rust, Python and Notebooks (Assignment, 180 minutes)

\- 3 ungraded labs (60 minutes each)

\- Module 4: Designing Data Processing Systems in Rust (13 hours)

\- 22 videos (Total 124 minutes)

\- 17 readings (Total 170 minutes)

\- 2 quizzes

\- Designing Data Processing Systems in Rust (Quiz, 30 minutes)

\- Final Course Quiz (Quiz, 30 minutes)

\- 2 assignments

\- Quiz-Getting Started with Rust Data Pipelines (Including ETL) (Assignment, 180 minutes)

\- Quiz-Using Rust and Python for LLMs, ONNX, Hugging Face, and PyTorch Pipelines (Assignment, 180 minutes)

\- 2 ungraded labs (60 minutes each)