Architectural Overview of Atlassian's Next-Generation Data Lakehouse

Explore an in-depth architectural overview of Atlassian's next-generation data lakehouse in this 41-minute conference talk. Discover how Atlassian is rebuilding their data infrastructure from the ground up, leveraging lessons learned from five years of Databricks experience. Learn about the layout of Databricks accounts, workspaces, and AWS accounts to create distinct environments. Understand the "workbench environment" concept that separates insights work from production pipelines. Gain insights into the benefits of Unity Catalog, Delta Lake, and Managed Tables. Examine strategies for creating and enforcing a consistent information architecture. Explore the advantages of making the data lakehouse declarative and limiting human interaction in the production environment. Delve into methods for supporting and governing machine learning workloads within this new architecture.

Architectural Overview of Atlassian's Next-Generation Data Lakehouse