Harvard AI Tutor Study: 194 Students Showed Twice the Learning Gains
A Harvard experiment involving 194 physics students found that those using a custom AI tutor showed roughly twice the learning gains of students in an active classroom setting.
A Harvard experiment involving 194 students found that a custom-built AI tutor was associated with learning gains roughly twice those recorded in an instructor-led active learning setting. The preliminary findings were reported by the Harvard Gazette in September 2024, while the final research was still awaiting publication at the time.
The study involved students enrolled in Harvard’s Physical Sciences 2 course. Over two weeks, participants experienced both an instructor-guided lesson and an AI-supported lesson, allowing researchers to compare the two approaches.
Students completed pre-tests and post-tests to measure learning, while researchers also assessed engagement, motivation, enjoyment and growth mindset. The preliminary analysis showed stronger learning gains among students using the AI tutor, along with higher reported engagement and motivation.
The system was not simply an unrestricted chatbot. Harvard lecturers Gregory Kestin and Kelly Miller designed it using structured prompts, pre-vetted conversations and course-specific content. The AI was intended to provide personalised feedback while allowing students to learn at their own pace.
Researchers said the approach could potentially free classroom time for problem-solving, projects and collaborative activities.
However, the experiment was not presented as evidence that AI should replace teachers. Kestin stressed that AI should support critical thinking rather than do the thinking for students.
Harvard researchers subsequently explored adapting the approach to other subjects, including multivariable calculus. The experiment highlighted how carefully designed AI tutoring could complement teaching when it is aligned with faculty expertise, structured feedback and specific learning objectives.
