Model Explanation and Prediction Exploration Using Spark ML

Explore techniques for explaining and predicting non-linear models using Spark ML in this 23-minute conference talk from Databricks. Delve into methods for clarifying black box models, explaining GLM and Random Forest predictions, and conducting what-if analyses. Learn how to implement a Spark library for model explanation and leverage a node.js library for browser-based applications. Discover real-world applications in healthcare data analysis, covering 50 billion predictions. Gain insights into explaining feature contributions for arbitrary population subsets and understand how changing feature values affects predictions. Master the tools to enhance model transparency and user understanding in machine learning applications.

Introduction
Agenda
Clarify Health
Accuracy
Explanation
Prediction Exploration
Random Forest