Public opinion polls ahead of the 2020 election were the most inaccurate in a generation, according to Josh Clinton, Abby and Jon Winkelried Chair and professor of political science, who recently served as chair of a special task force convened by the American Association for Public Opinion Research specifically to evaluate polling. The task force found that polling during the two weeks before the election overstated support for then-Democratic nominee Joe Biden by 3.9 percentage points, which was the largest polling error since 1980 when support for Democratic nominee Jimmy Carter was overestimated by 6 percentage points. The presidential election between Biden, the eventual winner, and incumbent president Donald Trump was much closer than polling had indicated. A report detailing the task force’s discoveries and conclusions is here.
“This discrepancy is across the board. It’s not a Republican polling problem or a Democratic polling problem,” said Clinton, who is also a senior election analyst for NBC News’ Decision Desk, which projects election winners as polls close in each state. “We found that, regardless of party and regardless of how they were done, sizable polling errors occurred for presidential races as well as senate and state-level races.”
As chair of the task force, Clinton offered his Vanderbilt students an opportunity to participate in meaningful and exclusive analysis of polls. In his spring and fall 2020 courses on elections, Clinton’s students helped analyze poll results for the task force report as they learned about previous elections. This unique opportunity to do real-life work gave students a taste of what professional pollsters do.
How did this happen?
Though the exact causes of the discrepancy are still being determined by the task force and other researchers, the analysis pointed toward several possible explanations. An unexpectedly large voter turnout may have contributed to polling error, given the historical level of turnout and the “perfect storm” of current events—a worldwide pandemic that caused a flagging economy and record levels of unemployment, a wide assortment of new voting methods, and increasing polarization in the political sphere. As a result, in 2020, in addition to the usual voters and newly eligible voters (those who just turned age 18 and newly naturalized citizens), there were many non-regular voters who cast ballots.
Another possible source of polling error may be the reluctance of Trump supporters to participate in polling. Leading up to the 2020 election, Trump had characterized polling as “fake” and biased against his campaign, which may have led his supporters to decline participation in polling. Thus the size of Trump’s support was not captured in pre-election polls, so the eventual number of voters who cast their ballot for Trump was larger than polls indicated. The task force’s analysis showed that the greatest errors occurred where Trump support was largest.
Be careful with poll numbers
Though polls are often used to predict support for a particular candidate or policy idea, Clinton said it is important to put the results in proper context. The “margin of error” for a poll, for example, is actually not a statement of polling accuracy. It indicates the range within which the poll numbers would fluctuate if the poll were carried out repeatedly.
“People sometimes think that just because there is a number attached that it must be precise. The reality is that there are a lot of errors that can accumulate in a single poll, based upon small decisions about what you assume about voters—which can actually have enormous consequences within a polarized electorate. A bunch of small errors can end up producing consequential polling errors—especially given how close recent elections are,” Clinton said. “One of the takeaways I hope people get from the report is that there are a lot of complexities that go into polling that are quite variable.”
So how should the public interpret poll numbers?
Clinton suggested that readers should take the margin of error stated on the poll, double it, and see whether the difference between the two candidates is within the doubled margin of error. If it is, then it is hard to conclude which candidate is actually ahead in the race.
“This should be empowering because it means if a poll shows the race to be close, your vote counts even more. And don’t be too quick to predict winners based on polls, given the difficulty involved. You can never be completely sure until the votes are counted and certified,” Clinton said.