Problem-Solving Strategies for Data Engineers

Data engineers are facing a wide variety of problems every day. Learn best practices on how to approach the typical challenges.

Introduction

• Introduction
• What you should know

1. Roles and Phases

• The data engineer
• All important data engineering project phases
• General challenges faced

2. Planning

• Understanding the status quo
• Collecting the right requirements
• Defining good KPIs

3. Design

• Keeping implementation efforts in mind
• Choosing the right architecture and framework
• Predicting costs and scaling better
• The right benchmarking of existing tools

4. Implementation

• Definition of work packages and responsibilities
• Risk assessment
• Testing the right parts
• Having a good documentation

5. Operations

• Approaches to monitoring
• Approaches to bug fixing
• Awesome training of staff, current and new
• How to improve processes

Conclusion

• Conclusion and outlook