Doubt Truth to Be a Liar - Non Triviality of Type Safety for Machine Learning

Explore the intricacies of type-safe feature engineering for machine learning in Scala through this comprehensive conference talk from Scala Days Chicago 2017. Delve into the challenges of designing type-safe feature vectors for general-purpose machine learning frameworks, where the types of engineered features are not known in advance. Learn about potential solutions utilizing Shapeless, Scala Macros, and Quasiquotes to achieve compile-time type safety. Follow along as the speaker demonstrates real-world applications, including predicting Titanic survival and building ML applications with Spark ML. Discover advanced techniques such as feature transformations with Implicit Classes, typed pipeline stages, and model training. Gain valuable insights into the future of type-safe machine learning development and walk away with key takeaways to enhance your Scala and ML projects.

Intro
A glimpse into the future
Machine Learning 101
Real-life ML
Salesforce Einstein
Predict survival on the Titanic
Building ML application with Spark ML
Types and interactions
Feature Types V2
Typed Features
Feature names from vals using Macros magic
Feature transformations with Implicit Classes
Features and Transformers
Typed pipeline stages
1-to-1 Transformer example
1-to-1 Estimator example
Model training (reduce)
Going forward with Optimus Prime
Key takeaways