Weight Standardization
Offered By: Yannic Kilcher via YouTube
Course Description
Overview
Explore a video explanation of the Weight Standardization paper, which introduces a novel approach to accelerate deep network training. Learn how this technique extends normalization to include network weights, leading to performance improvements and new state-of-the-art results when combined with GroupNorm. Discover the benefits of Weight Standardization in micro-batch training scenarios, where it outperforms other normalization methods. Understand how standardizing weights in convolutional layers smooths the loss landscape by reducing Lipschitz constants. Gain insights into the paper's methodology, results, and implications for various computer vision tasks, including image classification, object detection, and semantic segmentation.
Syllabus
Weight Standardization (Paper Explained)
Taught by
Yannic Kilcher
Related Courses
Neural Networks for Machine LearningUniversity of Toronto via Coursera 機器學習技法 (Machine Learning Techniques)
National Taiwan University via Coursera Machine Learning Capstone: An Intelligent Application with Deep Learning
University of Washington via Coursera Прикладные задачи анализа данных
Moscow Institute of Physics and Technology via Coursera Leading Ambitious Teaching and Learning
Microsoft via edX