The Data Janitor Returns

Explore a pragmatic approach to data-related challenges in this 48-minute conference talk from MLCon. Learn how to tackle problems with limited resources, navigate GDPR concerns, and cut through the hype surrounding deep learning. Discover insights on data integration within organizations, the evolution of roles, and practical problem-solving strategies based on real-world experience. Gain valuable knowledge on making impactful decisions, understanding the data pyramid, and implementing effective KPI definitions. Delve into topics such as Google Analytics, Tableau, AV testing, machine learning realities, data DevOps, and cloud-agnostic advice. Examine the pros and cons of various data engineering tools and technologies, including Hadoop, Spark, Presto, and Clickhouse. Acquire tips on finding skilled data engineers and integrating data with marketing efforts in this comprehensive, down-to-earth presentation for data professionals facing resource constraints.

Introduction
Challenges in the industry
Feedback
Hard curve over
Building the pyramid
Making programmatic KPI definitions
Google Analytics
Tableau
Project Adventures
The Pyramid
AV Testing
Funding
Leftovers
Machine Learning
Reality
One constructive approach
Data DevOps
Compression
Imagenet
A Cigar
Park
Data Engineering
Open Roots
Our original sin
The problem with distributed computing
Any advice
What makes a data engineer
Cloud agnostic advice
Better quality code
Transit
Hadoop
Showdown
Spark
Presto
Redshift vs Presto
Clickhouse
Google
Open Source
The Shepherd
The Bird of Prey
Making a decision
Finding data engineers
Marketing and data integration
Y2K
Outro