Synthetic Petri Dish - A Novel Surrogate Model for Rapid Architecture Search

Explore a novel approach to Neural Architecture Search (NAS) in this 33-minute video explanation of the paper "Synthetic Petri Dish: A Novel Surrogate Model for Rapid Architecture Search." Learn how researchers propose abstracting the essential parts of neural network architectures into smaller versions and evaluating them on custom-learned data points to predict performance more efficiently. Dive into the concept of the Synthetic Petri Dish model, which aims to accelerate the most expensive step of NAS by instantiating architectural motifs in very small networks and evaluating them using few synthetic data samples. Understand the motivation behind this approach, its implementation, and its potential to significantly improve the accuracy of predicting motif performance, especially when limited ground truth data is available. Follow along as the video breaks down the algorithm, explains the process of producing synthetic data, and demonstrates its application in experiments like the PTB RNN-Cell. Gain insights into how this innovative method could inspire new research directions in studying the performance of extracted model components in controlled settings.

- Intro & High-Level Overview
- Neural Architecture Search
- Predicting performance via architecture encoding
- Synthetic Petri Dish
- Motivating MNIST example
- Entire Algorithm
- Producing the synthetic data
- Combination with architecture search
- PTB RNN-Cell Experiment
- Comments & Conclusion