Speed is Not All You Need for Data Processing
Offered By: PyCon US via YouTube
Course Description
Overview
Explore the evolving landscape of data processing frameworks in this PyCon US talk by Kevin Kho and Han Wang. Delve into the comparison between traditional tools like Pandas and newer options such as Polars and DuckDB. Examine the common assumptions about speed and performance in data processing, particularly focusing on Pandas and Pandas UDFs in relation to native Spark. Through benchmarks across various data sizes, discover surprising insights that challenge conventional wisdom about tool selection. Learn why practitioners should prioritize tools that best serve their needs rather than solely focusing on perceived speed advantages. Gain valuable perspective on making informed decisions when choosing data processing frameworks for your projects.
Syllabus
Talks - Kevin Kho, Han Wang: Speed is Not All You Need for Data Processing
Taught by
PyCon US
Related Courses
Social Network AnalysisUniversity of Michigan via Coursera Intro to Algorithms
Udacity Data Analysis
Johns Hopkins University via Coursera Computing for Data Analysis
Johns Hopkins University via Coursera Health in Numbers: Quantitative Methods in Clinical & Public Health Research
Harvard University via edX