Uber's Batch Analytics Evolution from Hive to Spark

Explore Uber's strategic migration from Hive to SparkSQL in this 28-minute conference talk. Discover how Uber tackled the challenge of optimizing their batch analytics processes, which previously accounted for 40% of their multimillion-dollar ETL expenses. Learn about the development of automation features, including query transpilation, parallel execution, and a validation framework for data correctness and performance. Delve into the architecture of Uber's auto-migration framework, understand the challenges faced during the migration process, and gain insights into the solutions implemented. Senior Software Engineers Akshayaprakash Sharma and Kumudini Kakwani from Uber share their experiences and reveal the overall efficiency gains achieved through this large-scale migration effort.

Uber's Batch Analytics Evolution from Hive to Spark