Reinforcement Learning from Human Feedback (RLHF)

In this course we explore one corner of the expanding AI universe, and review some of the basic principles found in reinforcement learning from human feedback (RLHF), the technology underlying great AI tools such as ChatGPT, Bard, and more.

Have you ever wondered how tools like ChatGPT and Bard are able to generate great responses to the questions we pose? How they can respond to a prompt like “Plan a trip to Italy this fall and suggest great things to see,” and produce a response containing a full itinerary with places to see, the best time to visit, and the sites you shouldn't miss? In this course, Reinforcement Learning from Human Feedback (RLHF), you’ll gain the ability to understand what is going on behind the scenes to create responses to your prompts. First, you’ll explore why having all the information available is not enough to create a great response. Next, you’ll discover how we teach a machine learning model to handle all that data and craft a response that people like. Finally, you’ll learn how none of it is magic, just some really great engineering by some bright people. When you’re finished with this course, you’ll have the skills and knowledge of reinforcement learning with human feedback needed to understand how this great engineering works and produces its amazing results.

• Course Overview 1min
• Understanding Text-generative Applications 6mins
• What Is Wrong with the Pre-trained GPT Model? 5mins
• Supervised Fine-tuning 4mins
• Reward Model Training 11mins
• Fine-tuning via Reinforcement Learning 5mins
• Challenges and Limitations of RLHF 5mins