Data Engineering with dbt

Learn how to get started with setting up, running, and managing a dbt project.

Introduction

• Build your first dbt project
• Introducing the dbt project
• The project in GitHub Codespaces

1. Prepare Your Coding Environment

• Install the dbt Core via pip
• Install the dbt connector to DuckDB
• Install DuckDB via pip
• Update your requirements.txt file within your project

2. Prepare Your Database Environment

• Create your database file
• Import CSV data into your new database

3. Create a dbt Project

• Start your project with dbt init

4. Prepare Your dbt Environment

• Understand the dbt_project.yml file
• Create your profiles YAML file
• Connect your profiles and project YAML files

5. Your First dbt Model

• Create your first dbt model file
• Using the dbt CLI commands

6. Introduction to dbt Ref Function

• Create your dbt model utilizing ref
• Run your dbt models with the ref syntax
• View your dbt project data lineage

7. Implementing Medallion Architecture with dbt

• Planning your medallion architecture project
• Medallion architecture: Bronze data
• Medallion architecture: Silver data
• Medallion architecture: Gold data

8. Materialization of dbt Models

• Materialization in your dbt project
• Implement materialization in your dbt_project.yml file

9. Documentation as Code via dbt

• Further documentation via schema.yml file
• The docs_blocks.md file

10. Implementing Tests within Your dbt Project

• Creating custom singular tests
• Implementing tests within the schema.yml file

11. Deploying Your dbt Project

• Utilizing multiple dbt profiles
• Deploying with GitHub workflows

Conclusion

• Next steps