Data Science Foundations: Data Engineering

Discover the basics of big data with a data science expert. Learn about how to perform core data engineering tasks including staging, profiling, cleansing, and migrating data.

Introduction

• Welcome
• What you should know before watching this course
• Using the exercise files

1. Ecosystem Overview

• Data science system overview
• Star schema design overview
• Where does data engineering fit?
• Components of a good data pipeline
• Environment setup

2. Staging Data

• Loading and profiling data
• Data quality testing

3. Cleansing Data

• Adding data types
• Handling missing values
• Verifying addresses

4. Conforming Data

• Performing master data lookups
• Handling inferred members

5. Delivering Analytical Data Sets

• Loading the star schema
• Loading dimension tables
• Loading fact tables
• Creating views

Conclusion

• Next steps