Microsoft Azure Data Engineer Associate (DP-203) Cert Prep: 4 Monitor and Optimize Data Storage and Data Processing by Microsoft Press

Explore the concepts and skills required to monitor and optimize data storage and data processing to pass the Microsoft Azure Data Engineer Associate (DP-203) certification exam.

1. Monitor Data Storage

• Learning objectives
• Implement logging used by Azure Monitor
• Configure monitoring services
• Measure performance of data movement
• Monitor and update statistics about data across a system
• Monitor data pipeline performance
• Measure query performance

2. Monitor Data Processing

• Learning objectives
• Monitor cluster performance
• Understand custom logging options
• Schedule and monitor pipeline tests
• Interpret Azure Monitor metrics and logs
• Interpret a Spark Directed Acyclic Graph (DAG)

3. Tune Data Storage

• Learning objectives
• Compact small files
• Rewrite user-defined functions (UDFs)
• Handle skew in data
• Handle data spill
• Tune shuffle partitions
• Find shuffling in a pipeline
• Optimize resource management

4. Optimize and Troubleshoot Data Processing

• Learning objectives
• Tune queries by using indexers
• Tune queries by using cache
• Optimize pipelines for analytical or transactional purposes
• Optimize pipeline for descriptive versus analytical workloads
• Troubleshoot failed Spark jobs
• Troubleshoot failed pipeline runs

Conclusion

• Summary