YoVDO

Stateful Streaming with Apache Spark - Updating Decision Logic at Runtime

Offered By: Databricks via YouTube

Tags

Apache Spark Courses Data Engineering Courses Spark Streaming Courses Loyalty Programs Courses

Course Description

Overview

Learn how to implement a stateful streaming application with Apache Spark in this 25-minute conference talk from Databricks. Explore real-time decision logic for a multi-partner loyalty program, focusing on updating configurations at runtime without manual deployment. Discover techniques for reading external sources to update internal state, broadcasting state to all executors, and restoring state after restarts or failures. Gain insights into Spark Streaming, mutable broadcasts, and solving complex business challenges in data engineering.

Syllabus

Introduction
Problem Statement
Agenda
Example
Mutable Broadcast
Spark Streaming


Taught by

Databricks

Related Courses

内存数据库管理
openHPI
CS115x: Advanced Apache Spark for Data Science and Data Engineering
University of California, Berkeley via edX
Processing Big Data with Azure Data Lake Analytics
Microsoft via edX
Google Cloud Big Data and Machine Learning Fundamentals en Español
Google Cloud via Coursera
Google Cloud Big Data and Machine Learning Fundamentals 日本語版
Google Cloud via Coursera