Shannon Entropy and Information Gain

Explore the concepts of Shannon Entropy and Information Gain in this 21-minute video from Serrano.Academy. Dive into probability theory and information theory through engaging examples, including ball-picking scenarios and games. Learn how to calculate probabilities, work with products, and handle multiple classes. Examine different sequences and compare naive approaches with more sophisticated methods. Gain a comprehensive understanding of these fundamental concepts in machine learning and data science, culminating in the general formula for Shannon Entropy and Information Gain.

Shannon Entropy and Information Gain
What ball will we pick?
Quiz
Question
Game
Probability of Winning
Products
What if there are more classes?
Sequence 2
Sequence 3
Naive Approach
Sequence 1
General Formula