Machine Learning with Scala on Spark

Explore machine learning with Scala on Apache Spark in this 39-minute conference talk from Scala Days Berlin 2016. Discover the advantages of Spark.ml over older technologies and compare it to widely used frameworks in R and Python. Learn about the stages of building a predictive model, including exploration, data cleaning, feature engineering, and model fitting. Gain insights into Spark.ml's ease of use, productivity, feature set, and performance. Understand the new capabilities Spark.ml offers to data scientists and machine learning practitioners. Delve into topics such as DataFrame vs DataSet, Scala as the defacto ML language, hyperparameter tuning, ETL processes, and pipeline approaches. Explore the pros and cons of using Spark for machine learning, and get inspired to join the community in using and improving this rapidly maturing technology.

Introduction
Definition of Machine Learning
Why talk about Machine Learning
Not happening is bad
Prediction
Software Industries
Machine Learning
Why do people do this
Why is DSR doing this
Python
Scala
Scala vs Spark
Machine Learners
Python vs Scala
Kratt
Menial Work
Lack of Understanding
Spark
Tabriz
DataFrame vs DataSet
Scala as the defacto ML language
Problem with onboarding
How we assimilate the influx
Twitter
Hire a person
MOOCs
No spec by spec
When will it work
How does it feel
The point of Spark
Why Spark
Hyperparameter Tuning
Data Cleaning
ETL
Migration
Catching Errors
Scale
Picture
categorical variables
pipelines
pipeline approach
graphics staff
pros
disappointments
conclusion
thank you