YoVDO

Building a Real-Time Feature Store for Machine Learning at iFood

Offered By: Databricks via YouTube

Tags

Real-Time Data Processing Courses Machine Learning Courses Redis Courses Delta Lake Courses Spark Structured Streaming Courses

Course Description

Overview

Explore how iFood, Latin America's largest food tech company, built a real-time feature store using Databricks and Spark Structured Streaming. Discover the innovative data processing pipelines that power machine learning models for order completion time estimation, restaurant recommendations, and fraud detection. Learn about the architecture combining event stream processing, Delta Lake Table storage, and Redis low-latency access clusters. Gain insights into iFood's development processes for creating production-grade, reliable, and validated code. Understand the challenges and solutions in handling large-scale data processing for 26 million monthly orders from over 150,000 restaurants. Delve into topics such as two-step aggregation, sparse streaming, materialization jobs, and fading jobs. Uncover the lessons learned and positive outcomes from implementing this advanced feature store system.

Syllabus

Introduction
AI Food
Feature Store
Feature Store Architecture
TwoStep Aggregation
Sparse Streaming
Materialization Jobs
Flexibility
Fading Jobs
Lessons
Positive Outcomes


Taught by

Databricks

Related Courses

Conceptualizing the Processing Model for Azure Databricks Service
Pluralsight
Apache Spark for Data Engineering and Machine Learning
IBM via edX
Microsoft Azure Data Fundamentals: Explore data analytics in Azure
Microsoft via Microsoft Learn
Processing Streaming Data with Apache Spark on Databricks
Pluralsight
Apache Spark Project for Beginners: A Complete Project Guide
Udemy