Optimizing BigQuery Operations for Large-Scale Data Management

Discover strategies for optimizing BigQuery operations in this conference talk from Google Cloud Next 2024. Delve into Geotab's experience of leveraging BigQuery to process petabytes of data daily, supporting over 1,000 data pipelines and fostering a data-driven culture across 80% of their teams. Learn essential techniques for enhancing BigQuery's efficiency and cost-effectiveness in large-scale data operations. Gain valuable insights into refining data architecture, managing storage and compute costs, improving data literacy, and implementing effective slot organization. Explore the use of dashboards for operational health monitoring and discover new recommenders to further optimize BigQuery performance. Presented by Vinay Yerramilli and Muhammad Junaid Gill, this 37-minute session offers practical knowledge for data professionals seeking to maximize their BigQuery utilization.

Introduction
Presentation
Agenda
What is Geotab
Challenges at Geotab
Approach to optimization
Data Architecture
Inje Cost
Storage Cost
Storage Billing Model
Compute Slots
Invest in Data Literacy
Slot Organization
Dashboards
Operational Health
New recommenders