Neural Nets for NLP - Debugging Neural Nets

Learn to debug neural networks for natural language processing in this comprehensive lecture from CMU's Neural Networks for NLP course. Explore techniques for identifying and resolving issues in both training and testing phases. Discover how to assess model capacity, handle initialization challenges, optimize minibatching, and improve decoding processes. Gain insights into effective data analysis, quantitative evaluation methods, and strategies to prevent overfitting. Master the art of fine-tuning neural networks to achieve optimal performance in NLP tasks.

Intro
In Neural Networks, Tuning is Paramount!
A Typical Situation
Possible Causes
Identifying Training Time Problems
Is My Model Too Weak? . Your model needs to be big enough to learn
Be Careful of Deep Models
Initialization
Bucketing/Sorting
Debugging Minibatching
Debugging Decoding
Debugging Search
Look At Your Data!
Quantitative Analysis
Example: compare-mt
Reminder: Early Stopping
Loss Function, Evaluation Metric
Symptoms of Overfitting