Inserting brief quiz questions into an online lecture can boost learning and may reduce racial achievement gaps, even when students are tuning in remotely in a distracting environment.
That’s a main finding of our recent research published in Communications Psychology. With co-authors Dahwi Ahn, Hymnjyot Gill and Karl Szpunar, we present evidence that adding mini-quizzes into an online lecture in science, technology, engineering or mathematics – collectively known as STEM – can boost learning, especially for Black students.
In our study, we included over 700 students from two large public universities and five two-year community colleges across the U.S. and Canada. All the students watched a 20-minute video lecture on a STEM topic. Each lecture was divided into four 5-minute segments, and following each segment, the students either answered four brief quiz questions or viewed four slides reviewing the content they’d just seen.
This procedure was designed to mimic two kinds of instructions: those in which students must answer in-lecture questions and those in which the instructor regularly goes over recently covered content in class.
All students were tested on the lecture content both at the end of the lecture and a day later.
When Black students in our study watched a lecture without intermittent quizzes, they underperformed Asian, white and Latino students by about 17%. This achievement gap was reduced to a statistically nonsignificant 3% when students answered intermittent quiz questions. We believe this is because the intermittent quizzes help students stay engaged with the lecture.
To simulate the real-world environments that students face during online classes, we manipulated distractions by having some participants watch just the lecture; the rest watched the lecture with either distracting memes on the side or with TikTok videos playing next to it.
Surprisingly, the TikTok videos enhanced learning for students who received review slides. They performed about 8% better on the end-of-day tests than those who were not shown any memes or videos, and similar to the students who answered intermittent quiz questions. Our data further showed that this unexpected finding occurred because the TikTok videos encouraged participants to keep watching the lecture.
For educators interested in using these tactics, it is important to know that the intermittent quizzing intervention only works if students must answer the questions. This is different from asking questions in a class and waiting for a volunteer to answer. As many teachers know, most students never answer questions in class. If students’ minds are wandering, the requirement of answering questions at regular intervals brings students’ attention back to the lecture.
This intervention is also different from just giving students breaks during which they engage in other activities, such as doodling, answering brain teaser questions or playing a video game.
Why it matters
Online education has grown dramatically since the pandemic. Between 2004 and 2016, the percentage of college students enrolling in fully online degrees rose from 5% to 10%. But by 2022, that number nearly tripled to 27%.
Relative to in-person classes, online classes are often associated with lower student engagement and higher failure and withdrawal rates.
Research also finds that the racial achievement gaps documented in regular classroom learning are magnified in remote settings, likely due to unequal access to technology.
Our study therefore offers a scalable, cost-effective way for schools to increase the effectiveness of online education for all students.
What’s next?
We are now exploring how to further refine this intervention through experimental work among both university and community college students.
As opposed to observational studies, in which researchers track student behaviors and are subject to confounding and extraneous influences, our randomized-controlled study allows us to ascertain the effectiveness of the in-class intervention.
Our ongoing research examines the optimal timing and frequency of in-lecture quizzes. We want to ensure that very frequent quizzes will not hinder student engagement or learning.
The results of this study may help provide guidance to educators for optimal implementation of in-lecture quizzes.
The Research Brief is a short take on interesting academic work.
