Were the polls wrong? [View all]

[kurtcagle]

I am the Community Editor for Data Science Central, so looking at statistics is both my passion and my job.

I've been pondering the comments that many people have said here at DU about never trusting the polls or pollsters again, because they were so off. My answer is simple - no, the polls were actually pretty accurate.

Most polling looks at distributions - the variations among the way that people will behave, with the idea that you can make models that test a lot of different variables and that in turn make it possible to combine or simulate the likelihood of a particular event occurring.

This year has been a real test for many pollsters and modelers, such as Five thirty eight. Between Covid-19 and the ensuing lockdown, the economic impact of that, racial protests and the like, there were a lot of factors that went into the models that were simply not knowable beforehand, because we've not had a widescale pandemic in modern times (the last one of note was the Spanish Flu in 1918-1920). Several modelers all made the same point - Biden will probably win but it could be by 5 electoral votes or 200. Florida was likely a toss-up, and outside of Miami-Dade country the numbers were in line with expectations, the same with Ohio (note that Ohio and Florida seem to be increasingly in sync, perhaps because Florida is where Ohioans retire to).

Similarly, Georgia and Arizona (and quite possibly North Carolina) were considered as being slightly leaning towards Biden, in part because all three of them are seeing growing college-educated populations in critical tech and media hubs offsetting retirees, and all three had similar probability distributions because of that (note that this becomes clearer looking at Omaha, which went for Biden, it too is becoming known as a technical hub. If Nebraska had a winner-take-all system, this detail would have been lost, as most of the rest of the state is agricultural and rural.

The same point can be made for most of the rest of the electoral map. The map that's emerging now is remarkably close to what most pollsters were projecting, Biden taking the Pacific West (including Colorado). Texas and the Interior West went to Trump (though it was closer than expected in Texas), and so forth. If you read beyond the headline numbers, however, what also emerges (and what's not always obvious beforehand) was that the likelihood of the Democrats capturing the Senate was above 50%, but not by much. They picked up two and lost one, and there are three more outstanding due to run-offs, so it's still possible.

What data analytics tells you are probabilities. and moreover probabilities after all the dust has settled. With network television, people became conditioned (perhaps too conditioned) to believing that elections would be over by the time midnight rolled around, but in reality, elections take time, though the use of mail-in ballots and a relatively long lead time, along with rules about secrecy, mean that we're now moving away from that model, for the better I believe. Government is a deliberative process, because it requires that we think about the choices that we make. In the era of quick gratification, the flashy TV graphics and horse-race pacing of coverage may garner more eyeballs, but it robs us of the opportunity to choose the best candidates or propositions.

So, before you condemn pollsters and data scientists as charlatans, wait a bit and get a clearer picture of the whole results, not the results in the moment. In 2020, the results followed the polls quite well, but only if you don't interpret them with additional expectations (a second blue wave, as an example).