Inference with Torch-TensorRT Deep Learning Prediction for Beginners - CPU vs CUDA vs TensorRT

Explore deep learning prediction using Torch-TensorRT in this comprehensive tutorial video. Learn to accelerate inference speed by comparing CPU, CUDA, and TensorRT implementations. Set up the development environment with Docker and Nvidia tools, then dive into using PyTorch to load and utilize the ResNet50 neural network for image classification. Discover techniques for image preprocessing, batch processing, and interpreting model predictions. Implement and analyze benchmarks to compare performance across different hardware configurations. Follow along to trace models, convert to TensorRT, and optimize inference speed. Gain practical insights into deep learning deployment and performance optimization for beginners and intermediate practitioners alike.

- intro
- clone Torch-TensorRT
- install and setup Docker
- install Nvidia Container Toolkit & Nvidia Docker 2
- Torch-TensorRT container option #1
- Torch-TensorRT Nvidia NGC container option #2
- import Pytorch
- load ResNet50
- load sample image
- sample image transforms
- batch size
- prediction with ResNet50
- softmax function
- ImageNet class number to name mapping
- predict top 5 classes of sample image topk
- speed test benchmark function
- CPU benchmarks
- CUDA benchmarks
- trace model
- convert traced model into a Torch-TensorRT model
- TensorRT benchmarks
- download Jupyter Notebook
- HOW DID I MISS THIS???
- thanks for watching!