Revisiting Nearest Neighbors from a Sparse Signal Approximation View

Explore a Google TechTalk presented by Sarath Shekkizhar that delves into revisiting nearest neighbors from a sparse signal approximation perspective. Gain insights into alternative neighborhood definitions, focusing on non-negative kernel regression (NNK) as an improved and efficient approach. Learn about the interpretation of neighborhoods as sparse signal approximation problems and how this view can enhance graph-based signal processing and machine learning. Discover a k-means-like algorithm leveraging NNK for data summarization and outlier detection. Examine a graph framework for empirically understanding deep neural networks, providing insights into model similarities, differences, invariances, and generalization performance. Explore topics such as kernel similarity, local linearity, basis pursuit, and the geometry of kernel ratio intervals. Witness practical applications through experiments in label propagation and classification, and understand the potential of NNK-Means for detecting representational outliers.

Intro
Revisiting Nearest Neighbors from a Sparse Signal approximation view
What is a neighborhood?
Neighborhood definitions: Kernels (Similarity)
Neighborhood definitions: Local linearity
Interlude: Sparse Signal Approximation
Neighborhood = Sparse signal approximation
Alternative: Basis pursuit
Neighborhoods: Non-negative basis pursuit
Non-Negative Kernel regression (NNK)
Geometry: Kernel Ratio Interval (KRI)
Example
Label propagation using graphs
Experiments: Label propagation
Experiments: Classification
Neighborhoods Summary
Conventional Approach
Solution: NNK-Means
kMeans vs Dictionary learning
Case study: Detecting unseen data using NNK-Means (representational outliers)
NNK-Means atom use in each scenario
NNK-Means for Outlier Detection
NNK-Means Summary
What is deep learning?
Graph based view of deep learning
NNK interpolation at penultimate layer