Did Polling Really Fail to Spot Trump’s Rise?
Many Americans did a double take Tuesday night as the election results began to come in. In state after state, Donald Trump was consistently outperforming even the most recent polls on his way to what many considered a shocking victory. So why did the actual results of the 2016 presidential election look so different than the picture painted by pre-election polls?
First, let’s look at how far off the polls actually were. An average of nine major polls taken in the week or so before the election, as reported by fivethirtyeight.com, gave Clinton about a 3 point lead in the national popular vote. While it looks like Clinton may finish slightly ahead when the votes are counted, the actual popular vote cast Tuesday was essentially a tie. In some swing states, polling error was slightly larger, while in others it was slightly smaller. A similar average of polls had Clinton up 3.5 points in Pennsylvania, while Trump won the state by 1 point. In Michigan, the average actually showed the two candidates in a dead heat, and Trump again won by 1 point.
The actual error when comparing an average of polls to the results was somewhat small: typically just a few percentage points. But the error was systematic. Trump outperformed his projections virtually everywhere. It’s likely that a number of factors led to this error, potentially including:
- Underperformance of third-party candidates – While Gary Johnson, the Libertarian candidate, performed just about on par with his polls nationally, he often underperformed his polling numbers in swing states. He received almost 2 percent less of the vote than the polling average in Pennsylvania, and 1 percent in Michigan. It’s possible that some voters who identified as Johnson supporters in the polls either changed their mind before Election Day and cast their votes for Trump, or simply felt more comfortable telling pollsters they were voting for Johnson.
- Undecided voters — Virtually all polls before Election Day had at least a few percent of responders saying they were unsure for whom they would vote. Since “none of the above” is not an option on the ballot, these voters might have largely broken for Trump before Election Day.
- Systematic sampling error – Donald Trump was a unique candidate in recent history, and it’s very possible that typically used sampling techniques missed a small portion of his electorate. Coming on the heels of polls also underestimating Brexit, the British vote to leave the European Union, observers have speculated that polls tend to miss some populist support.
In addition to these polling issues, some of the perceived error may be due to our own perceptions when forecasting uncertain events. Every reputable poll reports sampling error along with their numbers, and many of today’s projections are made in terms of probability rather than certain outcomes. For instance, fivethirtyeight.com put Clinton’s win probability between 65 and 75 percent in the couple of weeks prior to the election. That translates to a 1-in-3 or -4 chance of a Trump victory. We perceive Trump’s victory as a shock, but with odds like that, it shouldn’t be. A 1-in-3 or -4 event isn’t all that unlikely.
Forecasting outcomes in complex systems, such as the sum total of the decisions of millions of human beings, is fraught with opportunities for error. Four years from now, we’ll still be watching the polls, but perhaps we’ll interpret the results with a bit more nuance and humility.
Click here to sign up for the Daily Economy weekly digest!
Max Gulker is a former Senior Research Fellow at the American Institute for Economic Research. He is currently a Senior Fellow with the Reason Foundation. At AIER his research focused on two main areas: policy and technology. On the policy side, Gulker looked at how issues like poverty and access to education can be addressed with voluntary, decentralized approaches that don’t interfere with free markets. On technology, Gulker was interested in emerging fields like blockchain and cryptocurrencies, competitive issues raised by tech giants such as Facebook and Google, and the sharing economy. Gulker frequently appears at conferences, on podcasts, and on television. Gulker holds a PhD in economics from Stanford University and a BA in economics from the University of Michigan. Prior to AIER, Max spent time in the private sector, consulting with large technology and financial firms on antitrust and other litigation. Follow @maxg_econ.Max Gulker
