YoVDO

Azure Data Engineer Associate (DP-203) Cert Prep: 4 Monitor and Optimize Data Storage and Data Processing

Offered By: LinkedIn Learning

Tags

Apache Spark Courses Data Engineering Courses User-Defined Functions Courses Data Pipelines Courses Azure Monitor Courses Query Tuning Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Learn about monitoring and optimizing data security and data processing on Azure and prepare to pass that domain of the Microsoft Azure Data Engineering (DP-203) exam.

Syllabus

Introduction
  • Course introduction
1. Monitor Data Storage and Data Processing
  • Implement logging used by Azure Monitor
  • Configure monitoring services
  • Measure performance of data movement
  • Monitor data system/pipeline/cluster performance
  • Measure query performance
  • Schedule and monitor pipeline tests
  • Interpret a Spark directed acyclic graph (DAG)
2. Optimize and Troubleshoot Data Storage and Data Processing
  • Rewrite user-defined functions (UDFs)
  • Handle skew in data and data spill
  • Tune shuffle partitions/pipelines
  • Optimize resource management
  • Tune queries by using indexers and cache
  • Troubleshoot a failed Spark job and pipeline run
Conclusion
  • Summary and next steps

Taught by

Noah Gift

Related Courses

CS115x: Advanced Apache Spark for Data Science and Data Engineering
University of California, Berkeley via edX Big Data Analytics
University of Adelaide via edX Big Data Essentials: HDFS, MapReduce and Spark RDD
Yandex via Coursera Big Data Analysis: Hive, Spark SQL, DataFrames and GraphFrames
Yandex via Coursera Introduction to Apache Spark and AWS
University of London International Programmes via Coursera