Democratizing AI - Back-Fitting End-to-End Machine Learning

Explore the democratization of AI and the implementation of end-to-end machine learning in this 50-minute conference talk from Strange Loop. Dive into LinkedIn's journey of embracing AI across various products and the challenges faced in onboarding new engineers, features, and modeling technologies. Learn about "Productive Machine Learning," a strategy aimed at doubling modeler efficiency while making AI accessible to engineers across LinkedIn. Discover the four key layers of this approach: Exploring and Authoring, Training, Deploying, and Running, along with the crucial elements of Health Assessment and Feature Marketplace. Gain insights into LinkedIn's AI infrastructure, including tools like Jupyter, Quasar, and Spark, as well as the importance of feature consistency and the training engine Photon-Connect. Understand the significance of health assurance in the model development cycle and the role of anomaly detection. Explore the impact of AI Academy in accelerating democratization and adoption of AI techniques across the organization. Take away valuable lessons on the importance of robust infrastructure and practices for serving AI at scale, beyond just focusing on machine learning algorithms.

Intro
Al for Linkedin products
Goal of Pro-ML
Pro-ML Layers
Organizationally
Exploring and Authoring - Jupyter
Exploring and Authoring - Quasar
Exploring and Authoring - Spark
Features and Algorithms
Feature Marketplace
Feature Access Consistency
Training Engine: Photon-Connect
How We Learn From Data
How We Make it Happen
Health Assurance Model Development Cycle
Anomaly Detection
Al Academy Accelerated Democratization
Adoption
Democratizing AI