Stanford Seminar - Thesis Defense: Smart Tutoring Through Conversational Interfaces

In the well-known two sigma problem introduced in 1984, Bloom found that students tutored by a one-on-one expert tutor achieved a learning outcome two standard deviations higher than those taught using traditional classroom methods. Since one-on-one tutoring is too costly to scale up to the majority of students, technology-based solutions have been suggested as promising solutions to simulate one-on-one human tutoring experiences. However, current automated computer-based tutors still primarily consist of learning activities with limited interactivity such as multiple-choice questions, review-and-flip flash cards, and listen-and-repeat practices. These tutors tend to be unengaging and thus their effectiveness typically relies on students' desire to learn. With recent advances in artificial intelligence (AI), we now have the potential to create conversation-based tutoring systems with the ability to provide personalized feedback to make learning more engaging and effective and eventually help bridge the gap between one-on-one human tutoring and computer-based tutoring.

In this dissertation, I present the design, development, and testing of four AI-based conversational tutoring systems that personalize learning for adults and children. For adult learning, I present two systems: QuizBot for helping college students learn factual knowledge and EnglishBot for tutoring second language learners in speaking English. For child learning, I present two systems embedding conversational tutors into narrative stories to supplement elementary school students' math learning: the first implemented using Wizard-of-Oz techniques and the second powered by online reinforcement learning algorithms. I conducted human evaluations with over 500 students using these tutoring systems to better understand how humans interact with AI in these educational systems. Our results show that, compared to current learning systems, conversation-based tutoring systems that leverage new natural language processing and reinforcement learning techniques to provide adaptive feedback can engage students more, motivate them to spend more time using tutoring systems, and improve student learning outcomes.

Introduction.
Bloom's two-sigma problem.
Limitations of prior computer-based tutoring work.
Research questions.
EnglishBot interface.
Traditional system: a listen-and-repeat system.
Narrative stories have mixed impacts on learning.
Key components of the learning system.
Math learning task.
RL tutoring chatbot design: observation space.
RL tutoring chatbot design: reward function.