YoVDO

Composable Data Processing with Apache Spark - Scaling Development and Error Handling

Offered By: Databricks via YouTube

Tags

Apache Spark Courses Big Data Courses Data Processing Courses Software Architecture Courses Data Engineering Courses Scalability Courses Data Pipelines Courses

Course Description

Overview

Explore a 28-minute conference talk on composable data processing with Apache Spark, presented by Databricks. Learn about the challenges of scaling Spark development and the consequences of isolated Spark apps. Discover SIP, an extensible plugin framework used in Adobe's Experience Platform for data processing, which addresses issues of resiliency, scalability, monitoring, and error handling. Dive deep into SIP's detailed error reporting and its improved user experience. Gain insights into parsing errors, conversions, and implementation challenges in the Adobe Data Platform context.

Syllabus

Intro
Adobe Data Platform
Implementation Challenges
Parsing Errors
Conversions


Taught by

Databricks

Related Courses

CS115x: Advanced Apache Spark for Data Science and Data Engineering
University of California, Berkeley via edX Big Data Analytics
University of Adelaide via edX Big Data Essentials: HDFS, MapReduce and Spark RDD
Yandex via Coursera Big Data Analysis: Hive, Spark SQL, DataFrames and GraphFrames
Yandex via Coursera Introduction to Apache Spark and AWS
University of London International Programmes via Coursera