Building the Foundations of an Intelligent, Event-Driven Data Platform at EFSA

Explore the development of an intelligent, event-driven data platform at the European Food Safety Authority (EFSA) in this 27-minute Databricks presentation. Learn how EFSA, in response to new EU regulations, implemented a large-scale digital transformation program to increase transparency in risk assessment processes. Discover the architecture of a modern data platform running on Azure and powered by Databricks, acting as a central control tower for data brokerage. Examine the two key patterns driving the platform: an Event Driven Architecture (EDA) for real-time event processing and a lakehouse architecture for comprehensive data management. Gain insights into the challenges faced, the implementation of an object-oriented framework, and the creation of the Open EFSA portal for public access to scientific data. Understand how this innovative approach has positioned EFSA at the forefront of data-driven risk assessment in the food safety domain.

Intro
EFSA's scientific risk assessment process
A digital transformation opportunity...
To help EFSA comply, we worked on 3 main pillars
Control tower analogy
Central knowledge base fed by events
Making it work...
Technical Challenges
Object oriented framework
Executing the non-value adding activities
Exports implemented
Code reusability through class methods
Retake Challenges