YoVDO

Unit Tests - Overcoming Challenges in Structured Streaming

Offered By: Databricks via YouTube

Tags

Data Engineering Courses Software Testing Courses Apache Spark Courses Databricks Courses Unit Testing Courses Continuous Deployment Courses Continuous Integration Courses Test-Driven Development Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Explore strategies for overcoming challenges in unit testing structured streaming jobs in this 31-minute conference talk by Bartosz Konieczny, a freelance data engineer. Learn how to improve control over your codebase and increase confidence in your CI/CD pipeline releases. Discover solutions to common pain points data engineers face when writing unit tests for streaming jobs, including how to control assertions for long-running processes and represent third-party data stores and APIs in code. While focused on streaming jobs, gain insights applicable to batch pipelines as well. Enhance your data quality and processing layer through improved unit testing techniques for data engineering projects.

Syllabus

Unit Tests: How to Overcome Challenges in Structured Streaming


Taught by

Databricks

Related Courses

CS115x: Advanced Apache Spark for Data Science and Data Engineering
University of California, Berkeley via edX Big Data Analytics
University of Adelaide via edX Big Data Essentials: HDFS, MapReduce and Spark RDD
Yandex via Coursera Big Data Analysis: Hive, Spark SQL, DataFrames and GraphFrames
Yandex via Coursera Introduction to Apache Spark and AWS
University of London International Programmes via Coursera