Iceberg REST Catalog: Benefits, Migration, and Extensions

Explore the benefits and implementation of Iceberg REST catalog in this 39-minute conference talk by Hongyue Zhang from Apple. Learn about the advantages of using Apache Iceberg's new open-source table format for ACID-compliant, petabyte-scale datasets. Discover the journey from Hive metastore to REST catalog, including migration strategies, lessons learned, and extended functionality beyond Apache Spark and Flink. Gain insights into integrating authentication and authorization plugins for access control, and understand how to expose files behind Iceberg tables through new APIs. By the end of this presentation, acquire the knowledge needed to initiate your own migration to the REST catalogue and leverage its improved security, extensibility, and language-agnostic design.

Intro
Presentation
Catalog
Hive History
Metadata Swap
Error message and exception
Iper catalog properties
REST catalog
REST API
Iceberg vs R Catalog
Iceberg Scan and Matrix
Migrations
API Latency
Benchmark Process
Community Contribution
Questions