Polls: Impossible to say why the 2020 polls went wrong
“We can rule out a few things, but it’s hard to prove what happened beyond certainty,” said Josh Clinton, a professor at Vanderbilt University and chair of the association’s 2020 election task force. “Based on what we know about polling, and what we know about politics, we have some prime suspects as to what might happen.”
These “prime suspects” will be uncomfortable with pollsters and those who rely on them, from political campaigns to news media. The most likely – if not certain – reason for the uneven poll results is that major groups of people do not answer polls in the first place.
Low response rates were a major concern pollsters for more than a decade. But the politicization of polling under Trump — including a feedback loop from the former president, who falsely denounced poll results he disliked as “spurious” or deliberately aimed at stifling enthusiasm for answering polls among GOP voters — appears to misrepresent the results with A segment of the Republicans refused to participate in the polls.
But the pollsters say they can’t be sure that’s the main reason, because you never know exactly who you’re not talking to.
This makes solving survey problems more difficult than diagnosing Four years ago, which focused mostly on adjusting polls to take into account Trump’s popularity with voters without college degrees and his corresponding weakness with college degrees.
“It seems reasonable to the task force that Republicans who participate in our polls are different from those who support Republican candidates who do not participate in our polls,” Clinton said. “But how do you prove that?”
The task force’s first task was to evaluate the performance of the 2020 public opinion polls. Based on this metric, the poll received a failing grade. While national elections were the worst in four decades, statewide elections in presidential, Senate, and state gubernatorial races have been as bad in the past as there have been records (20 years).
According to the report, national opinion polls for the presidential race conducted in the last two weeks of the election have fallen by an average of 4.5 percentage points, while opinion polls in the state have fallen by just over 5 points. Most of the error was one-way: Given the voting margin, national polls were very favorable to incumbent President Joe Biden with 3.9 points, and state polls were very favorable 4.3 points to Biden.
Most of the error came from underestimating Trump’s support, rather than overestimating Biden. Comparing the final election results to the poll numbers for each candidate, Trump’s support was underestimated by an average of 3.3 points, while Biden was overestimated by a point — turning what looked like a solid lead for Biden into a closer, if still crucial race . .
It wasn’t just the Trump effect, either. Opinion polls in Senate and gubernatorial elections fell by a larger margin: 6 points, on average.
“Within the state itself, voting error was often greater in Senate contests than in presidential contests,” the AAPOR report stated. “Whether candidates are running for president, senator or governor, ballot margins generally indicate that Democratic candidates will do better and that Republican candidates will perform worse on the final approved vote.”
No one methodology was superior to the others. According to the report, there were only “minor differences” whether the surveys were conducted by phone, online, or using mixed methodology, including text and smartphone apps – or whether they contacted voters randomly versus a list of registered voters. “Every method of interviewing and every method of sampling overestimated the Democratic-Republican margin relative to the final approved voting margin,” the report said.
After the 2016 election, the AAPOR autopsy blamed polling errors that year on a number of different factors. First, the organization said, a disproportionate number of undecided voters measured in polls flocked to Trump at the end of the race, giving him an advantage that is impossible to measure in advance.
But the mistake of 2020 can’t be blamed on late decision-makers: Only 4 percent of voters haven’t been behind a leading candidate in statewide polls in the last two weeks, and polls suggest late-deciding voters are evenly split Almost between Biden. And Trump.
Another problem in 2016, the report said – many pollsters failed to vote for education – was not to blame last year. Four years ago, several pollsters adjusted their results to get the right mix of voters by race and gender. But that missed a key dynamic emerging in the electorate: White voters with college degrees increasingly supported Democrats, while those who didn’t graduate from college quickly flocked to Republicans. Studies show that voters without college degrees are less likely to vote.
However, in 2020, the majority of state polls made adjustments to get more non-college voters in their polls. But they are still wrong.
Other 2016-style factors were also dismissed: Voters didn’t lie to the pollsters supporting them because of some sort of “Trump timid” theory (otherwise the errors wouldn’t be greater in downballot races). It wasn’t that one candidate’s supporters didn’t turn up to vote (as evidenced by the record-breaking turnout in last year’s race). The estimation of how many voters would vote early versus showing up on Election Day was not the cause either (opinion polls often allowed this split).
The report is clear about the reasons why the 2020 election was not missed, but it says: “It seems impossible to determine why polls were overstated in the Republican-Democrat margin compared to the approved vote with the available data.”
The most plausible – but still unproven – theory is that the voters that polls reach are fundamentally different from those who are not. Trump’s statements about “fake” or falsified polls only exacerbate this problem.
If voters who are more supportive of Trump are the least likely to participate in polls, the polling error can be interpreted as: Republicans,” the report reads. “Even if the correct proportion of self-identified Republicans were polled, differences in Republicans who responded and did not respond could result in the observed poll error.”
AAPOR isn’t the only organization struggling to figure out where things have gone wrong. A collaborative report by five of the largest polling firms for the Democratic campaign, released this spring, He said “no consensus emerged on a solution” to fix the bugs of 2020.
While explanations remain elusive, pollsters and their clients are making strenuous efforts to introduce changes to methodologies. Solicitation of survey respondents via text – or all-text surveys – is increasingly popular as fewer Americans are willing to take a 15-minute phone survey. Online polling continues to grow as well.
Public opinion polls commissioned by the media also change. A Wall Street Journal spokesperson confirmed to Politico that NBC News and The Wall Street Journal ended their more than 30-year ballot partnership late last year. The two news organizations have long worked with a pair of major bipartisan polling firms on regular telephone polls.
Without definitive answers about the reasons for missing out on 2020, pollsters are not sure they will be able to get it right in 2022, 2024 or later.
“Even seven months after the incident, you think you’ll be able to tell exactly what happened,” Clinton said.
“How sure are we that we can fix this in the future? Well, it’s not clear,” Clinton added. “We’ll have to wait and see – that’s not a particularly reassuring situation. But I think that’s the honest answer.”