YoVDO

Shannon Entropy and Information Gain

Offered By: Serrano.Academy via YouTube

Tags

Machine Learning Courses Information Theory Courses Probability Courses

Course Description

Overview

Explore the concepts of Shannon Entropy and Information Gain in this 21-minute video from Serrano.Academy. Dive into probability theory and information theory through engaging examples, including ball-picking scenarios and games. Learn how to calculate probabilities, work with products, and handle multiple classes. Examine different sequences and compare naive approaches with more sophisticated methods. Gain a comprehensive understanding of these fundamental concepts in machine learning and data science, culminating in the general formula for Shannon Entropy and Information Gain.

Syllabus

Shannon Entropy and Information Gain
What ball will we pick?
Quiz
Question
Game
Probability of Winning
Products
What if there are more classes?
Sequence 2
Sequence 3
Naive Approach
Sequence 1
General Formula


Taught by

Serrano.Academy

Related Courses

Introduction to Artificial Intelligence
Stanford University via Udacity Natural Language Processing
Columbia University via Coursera Probabilistic Graphical Models 1: Representation
Stanford University via Coursera Computer Vision: The Fundamentals
University of California, Berkeley via Coursera Learning from Data (Introductory Machine Learning course)
California Institute of Technology via Independent