Decision-Theoretic Planning to Control Crowdsourced Workflows

Explore decision-theoretic planning techniques for controlling crowdsourced workflows in this 52-minute seminar by Dan Weld from the Paul G. Allen School. Delve into the challenges of constructing effective workflows in crowd-sourcing labor markets and citizen science platforms. Learn about the application of probabilistic planning and reinforcement learning algorithms to optimize task performance. Discover the use of partially-observable Markov decision processes (POMDPs) for controlling voting and iterative improvement workflows. Examine decision-theoretic methods for dynamic workflow switching and their advantages over traditional A-B testing. Investigate a novel approach to crowdsourcing taxonomy construction and methods for optimizing labeled training data acquisition in machine learning applications. Gain insights into the future of crowdsourced workflow control and potential areas for further research in this field.

Introduction
Context
Crowdsourcing
Collective Assessment
Markov Decision Processes
Control of Simple Tasks
Taxonomy Generation
Lydias Algorithm
Task Routing
Crowdsourcing for NLP
Control Problem
Question Selection
Impact Sampling