Balancing Speed and Accuracy in Model Development

Explore the critical balance between speed and accuracy in model development through this insightful 25-minute conference talk by Ivan Popov at Conf42 Python 2024. Delve into factors impacting model performance, business implications, and real-world examples. Learn optimization strategies, including data quality assessment, preprocessing techniques, and feature selection using SHAP values. Discover how to identify inefficiencies with tools like Yappi and compare popular models such as XGBoost and LightGBM. Gain valuable insights on aligning model development with business objectives and making informed decisions in the machine learning process.

intro
preamble
data scientist at about & render, london, uk
today's talk
the essence of balance: speed vs accuracy
factors impacting accuracy and speed
the business impact of speed and accuracy
real-world examples
balancing act: speed, accuracy, and cost
strategic importance of the balance
how to understand business objectives
scenarios for ml-models
optimisation strategies
training data quality and quantity
what is a good dataset?
what is a bad dataset?
data pre-processing
how to find inefficiencies in data pre-processing?
yappi
most common inefficiencies
feature selection
shap values for feature selection
model selection
xgboost
lightgbm
how to choose the best option
a quick recap
thank you for your time!