Engagement Activity Delta Lake for Einstein Analytics and Sales Cloud Einstein

Explore the development of the engagement activity Delta Lake at Salesforce in this 51-minute tech talk. Discover how this platform supports Einstein Analytics and Sales Cloud Einstein by capturing and storing user engagement activities. Learn about ingesting data, implementing incremental reads, ensuring exact-once writes across tables, handling mutations with cascading changes, and normalizing tables in the data lake. Gain insights from Salesforce software engineers as they discuss the intricacies of building a high-volume, low-latency data pipeline for High Velocity Sales. Delve into topics such as data file hierarchy, atomic actions, metadata management, mutation operations, and stress testing. Understand the benefits of using Delta Lake for data lake updates, time travel capabilities, and snapshots in this comprehensive exploration of Salesforce's engagement activity platform.

Introduction
Agenda
Engagement Data Lake
Incremental Reads
Notification Table
Incremental Reach
Data File Hierarchy
Atomic Actions
Data Loss
Metadata
Batch ID
Happy Pass
Mutation Retry
Mutation Operations
Example
Data Shape
Mapping Table
Stress Test
Questions
Data lake updates
Kafka vs Spark
Time Travel
Snapshots
Callout
Why
Wrap Up