Scaling Privacy in a Spark Ecosystem

Explore the critical topic of privacy in data management through this 26-minute conference talk from Databricks. Learn why privacy has become a paramount concern in today's data landscape, balancing customer rights protection with business needs. Discover the challenges of scaling privacy in open data ecosystems and examine strategies for implementing agile data practices while maintaining privacy. Gain insights from CTO Don Bosco Durai of Privacera and Northwestern Mutual as they present a crucial privacy use case. Understand the differences between various privacy concepts and explore solutions for centralized access control, auditing, and reporting. Delve into the complexities of privacy management and learn how to scale privacy efforts effortlessly to meet business requirements in a Spark ecosystem.

Intro
Backgrounds
Why do we suddenly care about privacy?
What is the difference between these?
Examine strategies to scale agile data w/privacy
Challenges to that strategy
Privacy Challenges in Open Data Ecosystem
Centralized Access Control
Centralized Auditing and Reporting