Millions of people sign up for free online higher education courses offered by top-tier institutions, but only a small percentage of registrants earn a completion certificate. A new large-scale Vanderbilt study took an in-depth look at what factors contribute to student persistence and engagement in Massive Open Online Courses (MOOCs).
Brent J. Evans, assistant professor of public policy and higher education at Vanderbilt’s Peabody College of education and human development, led the study with colleagues at University of California, Irvine, and Stanford Graduate School of Education. Results are published in the Journal of Higher Education.
“A single MOOC can easily attract tens of thousands of registrants, but maybe half of those show up and interact even once,” Evans said. “That said, many thousands do complete the courses they sign up for. We wanted to find out what factors were indicators of persistence and engagement.”
The study examined a unique dataset of 44 MOOCs on the Coursera platform, comprised mostly of Stanford courses in science, technology, engineering and mathematics (STEM). The researchers evaluated 2.1 million student observations across 2,900 lectures to unearth critical patterns of enrollment, engagement, persistence and completion.
Some of the results are surprising. Students who signed up for courses a month or more ahead of time—presumably those highly motivated to take the course—were actually far less likely to participate than those who signed up just before the course began.
“We found there was this sweet spot for registering,” Evans said. “The early birds, for whatever reason, were less likely to show up. Those who signed up one to three weeks before the course began were the most likely to persist.”
The students with the highest probability of actually completing the course were those who agreed to take a pre-course survey. Those students watched 12 more lectures and were 12 percentage points more likely to earn a completion certificate than those who chose not to complete the survey.
“It was fascinating to discover that the student’s willingness to take the survey was the strongest predictor that they would complete the course,” Evans said.
When asked in the survey what their motivation was for taking the course, the majority of respondents cited relevance to their job or a substantial interest in the subject matter. However, those who said the prestige of the university was their highest motivating factor watched the highest percentage of lectures.
As far as engagement in the course material, lecture titles using the words “intro,” “overview” and “welcome” had a much higher rate of being watched than those that included summative words like “review” or “conclusion.” Videos labeled “exercise” had the largest negative association.
The length of the lecture seemed to have no impact on whether students chose to watch.
“Our hypothesis was that the length of a lecture might affect engagement,” Evans said. “But we found that students were just as likely to watch a five-minute video as a 20-minute video. The prevailing thought has been that breaking up lectures into shorter videos was more attractive to students, but we didn’t find that to be the case.”
Video lectures posted early in the week were more likely to be watched than those posted later in the week, and emails from the instructor to notify students of a new lecture did not induce them to watch.
“MOOCs represent a kind of new frontier,” Evans said. “We were surprised at some of these findings, but we believe this will provide insight into course design that could increase engagement in this important new educational setting.”
Collaborators on the study were Rachel B. Baker, assistant professor of educational policy at University of California, Irvine’s School of Education; and Thomas S. Dee, professor of education at the Stanford Graduate School of Education.