Protecting PII and PHI Data in Data Lakes with Column-Level Encryption

Learn how to implement column-level encryption to protect Personally Identifiable Information (PII) and Protected Health Information (PHI) in data lakes. Explore a comprehensive approach to data security, including design principles for column-level encryption, methods to safeguard PII data used in joins, techniques for authorized runtime decryption, and strategies for encryption key rotation. Discover how Northwestern Mutual leverages Fernet and AES encryption libraries, user-defined functions, and Databricks secrets to create a secure process for encrypting PII, limiting access to authorized personnel only. Gain insights into building a secure data lake that mitigates both internal and external data breach threats without duplicating data or metadata. The presentation covers the importance of data protection, recent breach examples, encryption in transit, and practical tips for implementation, concluding with a discussion on Business Intelligence and PII handling.

Introduction
Welcome
About Northwestern Mutual
Agenda
What is PII
How can the company lose data
Recent data breaches
Encryption in transit
Steps to do column level encryption
Everyone has access to the data
Architecture
bottleneck
tips tricks
more details
Questions
BI and PII