Introduction to Capsule Networks for Computer Vision

Explore capsule networks for computer vision in this 31-minute tutorial from the CVPR 2019 conference, presented by Sara Sabour from Google. Delve into sparse pattern recognition, clutter handling, and viewpoint equivariance. Learn about coordinate frames, vector formats, and object detection through part prediction agreement. Discover capsule network concepts, including squashed capsules, dynamic routing, and EM routing for Gaussian capsules. Examine the transform, agreement, and assignment steps in routing, and observe how capsule networks generalize viewpoints and handle constellations. Gain insights into this innovative approach to computer vision that addresses limitations of traditional convolutional neural networks.

Intro
Exercise
Sparse Pattern Recognition
Clutter
How can we handle different viewpoints?
Viewpoint Equivariance
What stays constant?
Coordinate frame
How to work in this vector format?
How can we detect objects?
How to detect objects? • An object exists if there is agreement between multiple part predictions
Agreement and Assignment
Capsule Network
Squashed Capsules: Agreement
Squashed Capsules: Assignment Dynamic Routing
New visual symbols for clarity
EM routing for Gaussian Capsules
Transform
Agreement (M step)
Assignment (Estep)
Routing in action
Viewpoint generalization
Constellations