Executive Guide to Human-in-the-Loop Machine Learning and Data Annotation

Explore the core concepts and key skills of human-in-the-loop machine learning, including how to successfully implement and manage a data annotation project.

Introduction

• Meet the human-in-the-loop

1. Introducing Human-in-the-Loop

• Human-in-the-loop is everywhere
• What is supervised machine learning?
• Defining human-in-the-loop
• The rise of human-in-the-loop

2. Common Use Cases

• Healthcare use cases
• Autonomous use cases
• Natural language use cases
• Reinforcement learning with human feedback (RLHF)

3. The Data Annotation Ecosystem

• The world of data annotation vendors
• Data annotation as a skill
• The ethics of data annotation
• Data annotation software vendors

4. Managing a Data Annotation Project

• Quality control
• Sampling
• Considerations when managing your own project
• Being data-centric

Conclusion

• Next steps