YoVDO

LLMOps: OpenVino Toolkit Quantize to 4int LLama 3.1 8B Inference on CPU

Offered By: The Machine Learning Engineer via YouTube

Tags

Machine Learning Courses Deep Learning Courses LLaMA (Large Language Model Meta AI) Courses Inference Courses OpenVINO Courses Model Compression Courses LLMOps Courses

Course Description

Overview

Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only!
Learn how to convert a LLAMA3.1 8B Instruct model to IR OpenVino format and quantize it to 4-bit integer precision in this 24-minute video tutorial. Discover the process of performing inference on CPU using the quantized model. Follow along with the provided GitHub notebook to gain hands-on experience in LLMOps, focusing on the OpenVino Toolkit for model optimization and deployment. Explore practical applications in data science and machine learning as you master techniques for efficient model compression and inference on CPU hardware.

Syllabus

LLMOps: OpenVino Toolkit Quantize to 4int LLama3.1 8B Inference CPU #datascience #machinelearning


Taught by

The Machine Learning Engineer

Related Courses

TensorFlow Lite for Edge Devices - Tutorial
freeCodeCamp Few-Shot Learning in Production
HuggingFace via YouTube TinyML Talks Germany - Neural Network Framework Using Emerging Technologies for Screening Diabetic
tinyML via YouTube TinyML for All: Full-stack Optimization for Diverse Edge AI Platforms
tinyML via YouTube TinyML Talks - Software-Hardware Co-design for Tiny AI Systems
tinyML via YouTube