Automated Pipeline for Large-Scale Neural Network Training and Inference

Explore an in-depth presentation on building a cloud-native and scalable machine learning training pipeline for managing thousands of anomaly detection models. Learn how Mist, a Juniper company, automates ML operations including data collection, model training, validation, deployment, and version control. Discover the implementation of confident deep bidirectional long-short term memory (BiLSTM) models for predicting uncertainties in anomaly detection across numerous WiFi networks. Gain insights into novel statistical models developed to address challenges in unsupervised anomaly detection workflows. Understand the hourly anomaly detection service and weekly training jobs utilizing technologies such as Secor, Amazon S3, Apache Spark, Apache Kafka, and Elasticsearch. Delve into the details of unsupervised confident deep multivariate models for automatic WiFi network issue detection, and learn how relative entropy is used to automate the training workflow. Benefit from shared lessons and insights on productizing and monitoring thousands of ML models for automated anomaly detection in large-scale networks.

Ebrahim & Jisheng - Automated Pipeline for Large-Scale Neural Network Training and Inference