Comprehensive View on Intervals in Apache Spark 3.2

Explore the evolution of intervals in Apache Spark in this 25-minute Databricks video. Gain insights into the pre-3.2 version implementation and upcoming changes, including ANSI SQL standard conformance. Delve into existing API issues and learn how new year-month and day-time interval types aim to resolve them. Master the Interval API usage in Spark SQL and PySpark while avoiding potential pitfalls. Discover techniques for constructing intervals from external types and efficiently saving/loading via Spark's built-in datasources. Cover topics such as calendar intervals, new catalyst types, ordering intervals, SQL intervals, daytime intervals, timestamp intervals, and external intervals. Enhance your understanding of interval types and their operations to optimize your Apache Spark implementations.

Introduction
Agenda
Calendar Intervals
New Types
Ordering Intervals
SQL Intervals
New Catalyst Types
Creating Intervals
Table of Operations
Daytime Intervals
Timestamp Interval
External Intervals
Interval Types
Summary