How EleutherAI Trains and Releases LLMs - Interview with Stella Biderman
Offered By: Weights & Biases via YouTube
Course Description
Overview
Explore the world of large language models (LLMs) in this insightful interview with Stella Biderman, Executive Director at EleutherAI and Lead Scientist - Mathematician at Booz Allen Hamilton. Delve into EleutherAI's history, the similarities and differences between LLMs, and learn how to choose the right model for specific outcomes. Gain valuable insights on reinforcement learning, pre-training, and fine-tuning techniques for LLMs. Discover the best GPUs for LLM training and EleutherAI's unique approach to developing these models. Examine the importance of interpretability and mechanistic details in LLMs, and understand the significance of memorization in these powerful AI systems.
Syllabus
Intro
EleutherAI History & Intro.
LLMs similarities and differences.
Choosing a model for desired outcome
Reinforcement learning: benefits and challenges
Pre-training and fine-tuning LLMs
Best GPUs for LLMs
EleutherAI’s unique LLM-training
Interpretability’s mechanistic details
LLM memorization understanding importance
Taught by
Weights & Biases
Related Courses
Computational NeuroscienceUniversity of Washington via Coursera Reinforcement Learning
Brown University via Udacity Reinforcement Learning
Indian Institute of Technology Madras via Swayam FA17: Machine Learning
Georgia Institute of Technology via edX Introduction to Reinforcement Learning
Higher School of Economics via Coursera