With Joe Biden appearing to have just barely eked out a tight win in key swing states, historic Republican gains in the House of Representatives, and a high likelihood that Republicans will ultimately hold the Senate, Americans who closely followed the polls are scratching their heads and wondering what happened.
How could the experts get this so wrong? Is the polling industry finished forever? What happened to the double-digit Democratic leads in the upper Midwest? The chance to turn Texas blue? Sending Susan Collins packing?
Meanwhile, defenders of the political polling industry, like Nate Silver of FiveThirtyEight, who has made a living amassing polling data into complicated probabilistic prognostications, insist that the results were not really that bad.
“In the end, polling averages will probably ‘call’ the winners of all but 1-3 states correctly,” Silver tweeted Wednesday, “along with the winner of the popular vote, which should wind up at Biden +4/+5. That’s not great. It’s better to look at ‘margins’ and some of the margins were off. … In the final accounting, the polls will have done mediocrely, but not terribly.”
And he doubled down on Thursday saying that if the “polls have called 48/50 states correctly and Biden’s won the popular vote by 5.3 points or something, I don’t think there’s a ‘polls blew it again!’ narrative.”
So why the feeling of betrayal from those who felt like they’d been assured of another “blue wave” by political pollsters?
Never mind that the average high schooler could probably “call” 40/50 states correctly without any polling at all, there is a fundamental disconnect about what polls are supposed to do.
The truth is that while those who conduct polls take great pains to explain the inherent uncertainty and limitations of what they do, all of their influence and job stability is based on the media and general public believing that they are the oracles of a mystical mathematical religion, a class of sacred priests who can turn consulting fees and an economics degree into psychological insights into the soul of the country and accurate predictions of the future.
“Polls miss by about 3 points in an average year. That’s what they missed by in 2016 and I think that’s where we’ll end up with this year, too, once all votes are counted,” Silver tweeted Thursday. But he is shouting into the wind.
Of course polls miss. The people who conduct them understand that. And to their credit, they often do try to explain how their results should and should not be interpreted. But cautious assessments of probability do not viral tweets make.
Polling margins-of-error do indeed capture some of the uncertainty embedded in a poll, but less well understood is the fact that even in a perfectly conducted poll, the margin of error only represents 95% of the possible outcomes. In any given poll there is a one-in-20 chance of a larger polling “miss.”
Multiply that across the nation’s 435 congressional representative races, and over 20 races are likely to miss their polls by more than the margin of error. And while averaging many polls together can reduce some of that risk, the natural phenomenon of pollster herding and the public’s inclination to amplify the cherry-picked results that confirm their priors are a toxic combination.
And ultimately, all this only holds in the case of methodologically perfect polls. The mathematical models used in statistics assume (and attempt to collect) truly random samples of the public. If there is some other unintended error in the sampling, for example, a “shy Trump voter” effect, or undersampling of noncollege voters, or simply oversampling of the kind of people who answer pollster phone calls, all bets are off.
And thus the downfall of the pollster priest class. People come to them for comfort, for answers in the storm of political uncertainty that surrounds election season. Scientific, expert answers. Answers they simply don’t have the ability to provide.
Friends, this isn’t the time to be complacent. If you are ready to fight for the soul of this nation, you can start by donating to elect Joe Biden and Kamala Harris by clicking the button below.
Thank you so much for supporting Joe Biden’s Presidential campaign.