Speed is Not All You Need for Data Processing
Offered By: PyCon US via YouTube
Course Description
Overview
Explore the evolving landscape of data processing frameworks in this PyCon US talk by Kevin Kho and Han Wang. Delve into the comparison between traditional tools like Pandas and newer options such as Polars and DuckDB. Examine the common assumptions about speed and performance in data processing, particularly focusing on Pandas and Pandas UDFs in relation to native Spark. Through benchmarks across various data sizes, discover surprising insights that challenge conventional wisdom about tool selection. Learn why practitioners should prioritize tools that best serve their needs rather than solely focusing on perceived speed advantages. Gain valuable perspective on making informed decisions when choosing data processing frameworks for your projects.
Syllabus
Talks - Kevin Kho, Han Wang: Speed is Not All You Need for Data Processing
Taught by
PyCon US
Related Courses
DuckDB - High-Performance SQL Queries on Pandas Dataframe - PythonSamuel Chan via YouTube New Feature - What the Fibers Extension Can Do for You
International PHP Conference via YouTube Getting Started with PHP-FFI - Introduction to Foreign Function Interface
International PHP Conference via YouTube DuckDB - Bringing Analytical SQL Directly to Your Python Shell
EuroPython Conference via YouTube OLAP on Cassandra Data with Arrow, Flight SQL, ADBC, and DuckDB
Linux Foundation via YouTube