Stateful Streaming with Apache Spark - Updating Decision Logic at Runtime
Offered By: Databricks via YouTube
Course Description
Overview
Learn how to implement a stateful streaming application with Apache Spark in this 25-minute conference talk from Databricks. Explore real-time decision logic for a multi-partner loyalty program, focusing on updating configurations at runtime without manual deployment. Discover techniques for reading external sources to update internal state, broadcasting state to all executors, and restoring state after restarts or failures. Gain insights into Spark Streaming, mutable broadcasts, and solving complex business challenges in data engineering.
Syllabus
Introduction
Problem Statement
Agenda
Example
Mutable Broadcast
Spark Streaming
Taught by
Databricks
Related Courses
Big Data EssentialsA Cloud Guru Big Data
University of Adelaide via edX Advanced Data Science with IBM
IBM via Coursera Amazon EMR Getting Started (Indonesian)
Amazon Web Services via AWS Skill Builder Analisar e preparar dados com o Amazon SageMaker Data Wrangler e o Amazon EMR (Português (Brasil)) | Lab - Analyze and Prepare Data with Amazon SageMaker Data Wrangler and Amazon EMR (Portuguese (Brazil))
Amazon Web Services via AWS Skill Builder