Avoiding Catastrophe - Active Dendrites Enable Multi-Task Learning in Dynamic Environments
Offered By: Yannic Kilcher via YouTube
Course Description
Overview
Explore a comprehensive video review of the paper "Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments." Delve into the challenges of catastrophic forgetting in multi-task and continual learning, and discover how biological neural networks inspire solutions. Learn about active dendrites and their application in deep learning to combat forgetting while maintaining sparsity. Examine experiments in multi-task learning, continual learning, and adaptive prototyping. Analyze the algorithm's inner workings, compare it to larger networks, and explore its relationship with attention mechanisms. Gain insights into how biologically-inspired architectures can address dynamic scenarios that traditional artificial neural networks struggle with.
Syllabus
- Introduction
- Paper Overview
- Catastrophic forgetting in continuous and multi-task learning
- Dendrites in biological neurons
- Sparse representations in biology
- Active dendrites in deep learning
- Experiments on multi-task learning
- Experiments in continual learning and adaptive prototyping
- Analyzing the inner workings of the algorithm
- Is this the same as just training a larger network?
- How does this relate to attention mechanisms?
- Final thoughts and comments
Taught by
Yannic Kilcher
Related Courses
Active Dendrites Avoid Catastrophic Forgetting - Interview With the AuthorsYannic Kilcher via YouTube Supermasks in Superposition - Paper Explained
Yannic Kilcher via YouTube What Kind of AI Can Help Manufacturing Adapt to a Pandemic
Open Data Science via YouTube Rethinking Architecture Design for Data Heterogeneity in FL - Liangqiong Qu
Stanford University via YouTube Continual Learning and Catastrophic Forgetting
Paul Hand via YouTube