Digital focus groups to assess presidential ads weekly
In an election season that will shatter the record for money spent on a presidential campaign, political scientists at Vanderbilt and UCLA have created SpotCheck, a new approach for assessing political ads using internet-based surveys.
“We now can present evidence as opposed to speculation about the impact of a political spot,” says John Geer, the Gertrude Conaway Vanderbilt Professor of Political Science at Vanderbilt, who is partnering with Professor of Political Science Lynn Vavreck at UCLA on this new approach. “That will be invaluable as this presidential campaign goes forward.”
Total spending in the presidential campaign could get as high as $5 billion. That’s two and a half times what was spent just four years ago ($1.8 billion). Much of this money will be spent on television advertising, and yet little is known about the impact of these ads during the campaign.
Journalists and pundits offer their best guess about the ads’ impact — but they remain just that: guesses. [lquote]SpotCheck allows average citizens to evaluate presidential campaign ads in real time.[/lquote] In other words, we can move past guesswork and let the public directly assess political ads.
Each week between now and election day, SpotCheck will gather data from a representative sample of 1,000 people randomly assigned to watch and rate two advertisements selected by Vavreck and Geer. Internet research firms YouGov, G2 Analytics, and SageEngage will collect the data. The general public can visit the SpotCheck site to watch the ads, see the rankings of all the ads, explore the reactions from each ad in the study, and read short assessments from experts.
The experiments allow people to click a “bell” when they are pleased with what they are seeing or hearing and a “buzzer” when displeased. 
Afterward, they respond to a series of questions that further gauge their reaction to the ads. Rather than feeding the conversation about whether candidates’ ads are factually accurate, the project checks ads in a different way — to learn whether they have any chance of actually persuading voters.
“We are trying to measure two things: people’s emotional engagement with the ads and the effectiveness of the ads in terms of boosting a candidate’s favorability ratings relative to the opponent’s ratings,” Vavreck says. “We are interested in understanding which ads are the most persuasive and why.”
Vavreck and Geer have designed a formula for each ad that combines evaluations of emotional engagement and effectiveness to tally an overall score for each ad: the SpotCheck Score. The most effective ads to date include Bernie Sanders’ ad “America” that relies on a Simon and Garfunkel song and an ad from Our Principals PAC in which women read verbatim quotes of comments Donald Trump has made about women. The latter was the highest-scoring ad SpotCheck has tested to date.
New results will appear each week. These data can be found on the SpotCheck website. Readers can also watch the ads and take the survey themselves. “We don’t want expert raters,” Vavreck says. “We want regular people.”
With the amount of money being spent and with the technology available, SpotCheck provides a chance to democratize the narrative surrounding political ads and, in so doing, provide systematic evidence to advance the conversation of this ever dynamic presidential campaign.
Visit SpotCheck at vu.edu/spotcheck.