By Jenna Somers
It’s often hardest to collect data when you need it most, according to Elizabeth Zechmeister, Cornelius Vanderbilt Chair and professor of political science and director of the Latin American Public Opinion Project. When data is needed the most—when it is most informative—is often during times of civil unrest and national crises, such as a pandemic.
LAPOP’s research efforts never stopped when Vanderbilt closed labs in mid-March due to COVID-19. Instead, Zechmeister’s team pivoted to exclusively remote work and tackled the monumental task of overhauling research protocols to acquire data that reveals new insights about the pandemic’s effects on democracy.
LAPOP conducts the AmericasBarometer, the most expansive survey of democratic public opinion and behavior in the Western Hemisphere. During the 2018-19 round of the survey, Haiti experienced waves of civil unrest. While face-to-face interviews remain the gold standard for public opinion research in Latin America and the Caribbean, the situation in Haiti posed too great a risk for in-person interviews, so the data could not be included. Zechmeister and her colleagues knew they needed to devise another high-quality data collection method.
LAPOP embarked on that endeavor in January 2020, hosting a learning summit on Haiti and public opinion research, including a focus on phone surveys. At the time, LAPOP was preparing to manage a round of surveys in Haiti, Peru, Mexico and Ecuador, and they had established new connections with partners in Haiti who had the capacity to conduct phone surveys. Little did they know that their groundwork for doing these surveys would pay dividends once the COVID-19 pandemic hit.
In April, Haiti became the first country LAPOP surveyed with the new phone questionnaire protocol. Based on anecdotal accounts in the survey research community, the public seemed almost exclusively interested in discussing the pandemic and was reticent to receive questions on anything else, so LAPOP designed an experiment to test whether broad focus on the pandemic was affecting the quality of survey research on democratic public opinion.
Half of the respondents received 10 questions related to the pandemic at the start of the survey, and half received those questions at the end. “We found some evidence to suggest that if we didn’t start with questions on the pandemic, respondents were more likely to disengage and to end the interview,” Zechmeister said. “That influenced our design of the surveys moving forward; for the time being, we begin surveys by asking a couple questions about people’s experiences with, and evaluations of, the pandemic.”
In Haiti, LAPOP also tested whether citizens were inclined to rally around the incumbent administration during a crisis. Responses seemed to correlate with whether participants received the pandemic questions first or not. “In the version of the survey in which we raised the salience of the pandemic in the first 10 questions, people’s support for the incumbent president is higher than in the case where we don’t,” Zechmeister said. “So there is some evidence suggesting a tendency to rally around the executive in the early stage of the pandemic.”
Furthermore, they found in Haiti, Peru and Mexico that people are more tolerant of postponing elections in times of public health emergencies, such as the pandemic. “People’s support for democracy and democratic practices has been declining in the region since 2014,” said Zechmeister, “so we feel it’s important to consider what the pandemic is doing to people’s commitment to democracy. We’ve had similar conversations in the U.S. too. I don’t think it’s a sign that people are turning their backs on democracy in an absolute sense, but political scientists define democracy as a system that holds regular elections, so once you move into an area where you’re willing to postpone elections, at the very least your democracy is under stress.”
Zechmeister attributes the decline in support for democracy in Latin America and the Caribbean to a sense that the system has not been living up to its ideals and delivering good governance. Perceptions of corruption and poor performance have increased, and responses to the pandemic reflect even more poor performance. According to Zechmeister, it is still too early to know the secondary effects of the pandemic, but she worries that it will exacerbate dissatisfaction with democracy in the region.
As arduous as it has been for LAPOP to redesign research protocol, they had already intended to spend this year innovating—they just didn’t know they would be innovating around a pandemic. They worked non-stop throughout the spring and summer to learn about technology and best practices for phone surveys and then alter their survey design.
Managing the technological changes was one of the most challenging aspects of the shift since the methodology of phone surveys is much different than the methodology of face-to-face surveys. For efficiency and accurate population sampling, LAPOP needs to use random-digit dial technology, so they have been working with regional partners and consumer marketing firms across the Americas to increase this capacity. Nonresponse rates are significantly higher in phone surveys than in face-to-face surveys, so establishing an automated system in each country was a monumental, but essential, task.
Even when interviewers could reach a respondent, people were typically only willing to speak for 20–25 minutes compared to an average of 45 minutes for face-to-face interviews. So LAPOP downsized from about 160 questions to 60 for the phone interviews. They also modified how they asked questions because they couldn’t use their typical visual aids for face-to-face interviews. Lastly, they made post-survey adjustments to account for segments of the population who do not own cell phones. While cell phones are prevalent in Latin America and the Caribbean, people of lower socio-economic status are less likely to own them.
As difficult as it has been to collect data on democratic public opinion during the pandemic, LAPOP has risen to the challenge when it was needed the most. Their perseverance has led to critical insights on the pandemic’s effects on democracy, and they have held briefings on their findings with regional partners, the United States Agency for International Development and the U.S. ambassador to Haiti. Furthermore, their experience with managing phone surveys in Haiti, Peru, Mexico and Ecuador has enhanced processes as they gear up for the AmericasBarometer in 2021.
“That our team transitioned so well into remote work while altering survey protocol speaks volumes about how adaptable and nimble we are,” Zechmeister said. “Thanks to the steadfast dedication of the LAPOP team, we have a new toolkit to meet the challenges to come and to support our partners in the international community well beyond the pandemic.”
This story is part of a series highlighting Vanderbilt University researchers who have returned to in-person research activities on or off campus. More than 3,000 Vanderbilt research personnel have returned to in-person research activities through the Research Ramp-up process spearheaded by the Ad-Hoc Research Ramp-up Working Group and the Office of the Vice Provost for Research.