LLMOps: OpenVino Toolkit - Quantizing LLama3.1 8B to 4int and CPU Inference
Offered By: The Machine Learning Engineer via YouTube
Course Description
Overview
Explore the process of converting a LLAMA3.1 8B model to OpenVino IR format and quantizing it to 4-bit integer precision in this 19-minute video tutorial. Learn how to perform inference on CPU using the quantized model. Access the accompanying notebook on GitHub to follow along with the step-by-step instructions for model conversion, quantization, and inference using the OpenVino Toolkit. Gain practical insights into LLMOps techniques for optimizing large language models for efficient deployment on CPU hardware.
Syllabus
LLMOps: OpenVino Toolkit quantizar 4int LLama3.1 8B e Inferencia CPU #datascience #machinelearning
Taught by
The Machine Learning Engineer
Related Courses
Intel® Edge AI Fundamentals with OpenVINO™Intel via Udacity tinyML Vision Challenge - Intel-Luxonis DepthAI Platform Overview
tinyML via YouTube Machine Learning in Fastly's Compute@Edge
Linux Foundation via YouTube End-to-End AI Developer Journey with Containerized Assets Using Intel DevCatalog and DevCloud
Docker via YouTube Accelerate Your Deep Learning Inferencing with the Intel DL Boost Technology
EuroPython Conference via YouTube