Uber's Batch Analytics Evolution from Hive to Spark
Offered By: Databricks via YouTube
Course Description
Overview
Explore Uber's strategic migration from Hive to SparkSQL in this 28-minute conference talk. Discover how Uber tackled the challenge of optimizing their batch analytics processes, which previously accounted for 40% of their multimillion-dollar ETL expenses. Learn about the development of automation features, including query transpilation, parallel execution, and a validation framework for data correctness and performance. Delve into the architecture of Uber's auto-migration framework, understand the challenges faced during the migration process, and gain insights into the solutions implemented. Senior Software Engineers Akshayaprakash Sharma and Kumudini Kakwani from Uber share their experiences and reveal the overall efficiency gains achieved through this large-scale migration effort.
Syllabus
Uber's Batch Analytics Evolution from Hive to Spark
Taught by
Databricks
Related Courses
CS115x: Advanced Apache Spark for Data Science and Data EngineeringUniversity of California, Berkeley via edX Big Data Analytics
University of Adelaide via edX Big Data Essentials: HDFS, MapReduce and Spark RDD
Yandex via Coursera Big Data Analysis: Hive, Spark SQL, DataFrames and GraphFrames
Yandex via Coursera Introduction to Apache Spark and AWS
University of London International Programmes via Coursera