Video Analytics for Football Games

Explore video analytics for football games in this Devoxx conference talk by Sven Degroote. Dive into a use case utilizing Apache Beam to analyze and process near-real-time football game stream feeds. Learn how to determine events such as game start, team detection, player tracking, and ball tracking, while performing analytics on video duration, ball possession, and score. Discover the implementation of Apache Beam Dataflow runner with Python SDK to create streaming pipelines, using sliding windows to chunk video frames for machine learning model input. Understand the deployment of ML models on GPUs via TF-serving on Kubernetes, and the visualization of features using Google Cloud Bigtable. Gain insights into the project's architecture, including Google Cloud Pub/Sub, Google Kubernetes Engine, and the challenges faced in player detection, background subtraction, and coordinate transformation. Learn about leveraging managed services, pipeline deep dives, and testing in stream mode to create an efficient video analytics system for football games.

Intro
PLAYER AND BALL CAN BE DETECTED PER FRAME
EVENT DETECTION REQUIRES SEQUENCE OF FRAMES
PROJECT CONTEXT
THE PROBLEM LANDSCAPE
TWO SOLUTION PARTS
THE DATA FACTS
LEVERAGE THE MODEL TO SPEED UP THE LABELING
THE MODELS
PLAYER DETECTION: FIELD TRANSFORM
A WALK IN FEATURE SPACE
SUBTRACT BACKGROUND TO REMOVE THE NOISE
COORDINATES AS UNLOCKED DOWNSTREAM FEATURE
START OF GAME MODEL BEATS THE OTHER GOAL MODELS (FOR NOW)
SOLUTION ARCHITECTURE
ABOUT APACHE BEAM
THE SOLUTION LANDSCAPE
FROM HLS TO JPEG
FULLY LEVERAGE MANAGED SERVICES
LEVERAGE THE BEAM MODEL FOR PROCESSING
WHERE THE DATA CRUNCHING HAPPENS
PIPELINE DEEP DIVE
LEVERAGE THE INTERNAL LOAD BALANCER OF GKE TO GET PREDICTIONS
DEWARPING THE BOUNDING BOXES TO GET COORDINATES
TEAM DETECTION WITHOUT BACKGROUND SUBTRACTION
DUMPING THE PREDICTIONS TO BIGTABLE
LEVERAGE THE BEAM MODEL TO WINDOW THE DATA
RESPECT THE BEAM MODEL TO GET DESIRED PARALLELIZATION
TEST IN STREAM MODE