Mistral 7B: Architecture, Evaluation, and Advanced Techniques
Offered By: Trelis Research via YouTube
Course Description
Overview
Explore the capabilities and architecture of Mistral 7B in this 19-minute video tutorial. Dive into the model's design, setup process on Runpod, and comprehensive evaluation through various tests including random sequence reversal, code generation, passkey retrieval, and fine-tuning. Gain insights into the model's performance and understand advanced concepts like Grouped Query Attention and Sliding Window Attention. Access additional resources including a comparison notebook, Runpod setup guide, and a supervised fine-tuning tutorial to enhance your understanding of this powerful language model.
Syllabus
Intro
Video Overview
Mistral 7B architecture and design
Runpod setup
Mistral 7B Evaluation
Test 1: Random sequence reversal
Test 2: Code generation
Test 3: Passkey retrieval
Test 4: Fine-tuning
Evaluation Summary
EXTRA: Grouped Query Attention
EXTRA: Sliding Window Attention
Taught by
Trelis Research
Related Courses
Zephyr 7B Beta - Comparing a 7B LLM with 70B ModelsVenelin Valkov via YouTube Fine-Tuning a Local Mistral 7B Model - Step-by-Step Guide
All About AI via YouTube Personalizando LLMs: Guía para Fine-Tuning Local de Modelos Open Source en Español
PyCon US via YouTube Full Fine-Tuning vs LoRA and QLoRA - Comparison and Best Practices
Trelis Research via YouTube GPT-4 vs Open Source LLMs: Epic Rap Battles Test Creativity with AutoGen
Data Centric via YouTube