Collective Intelligence for Deep Learning - A Survey of Recent Developments

Explore a comprehensive survey of recent developments in collective intelligence for deep learning in this 28-minute conference talk by David Ha from Google Brain. Delve into the historical context of neural network research's involvement with complex systems and discover how principles of collective intelligence are being incorporated into modern deep learning methods. Learn about key concepts such as self-organization, emergent behavior, swarm optimization, and cellular systems, and their potential to address fundamental issues in current deep learning models. Examine active research areas that combine collective intelligence with deep learning to enhance robustness, adaptability, and flexibility. Gain insights into the bidirectional flow of ideas between complex systems and deep learning communities, and understand how modern deep learning models are advancing complex systems research. Discover topics including cellular neural networks, neural cellular automata, self-assembling reinforcement learning agents, self-attention mechanisms, sensory substitution, and multi-agent learning with larger population sizes.

Advances in Deep Learning
Modern Deep Learning requires Engineering
Cellular Neural Networks (1988)
Neural CA for Self-Classifying MNIST Digits
Neural CA for Minecraft
Self-Assembling Reinforcement Learning Agents
Self-Attention and Self-Organization for adapting to a changing observation space.
Sensory Substitution
Unexpected Generalization Results
Multi-Agent Learning: Larger Population Size (2018)