538: The Polls Were Historically Accurate In 2022

[brooklynite]

Let’s give a big round of applause to the pollsters. Measuring public opinion is, in many ways, harder than ever — and yet, the polling industry just had one of its most successful election cycles in U.S. history. Despite a loud chorus of naysayers claiming that the polls were either underestimating Democratic support or biased yet again against Republicans, the polls were more accurate in 2022 than in any cycle since at least 1998, with almost no bias toward either party.

Of course, some pollsters were more accurate than others. And today, we’ve updated the FiveThirtyEight pollster ratings to account for each pollster’s performance in the 2022 cycle. Our ratings are letter grades that we assign to each pollster based on historical accuracy and transparency. (You can read exactly how we calculate pollster ratings here.) They’re one of many tools you should use when deciding how much stock to place in a poll.

Before we reveal the best- and worst-rated pollsters, let’s start with our regular review of polling accuracy overall. We analyzed virtually all polls conducted in the final 21 days1 before every presidential, U.S. Senate, U.S. House and gubernatorial general election, and every presidential primary, since 1998,2 using three lenses — error, “calls” and statistical bias — to conclude that 2022 was a banner year for polling.

In our opinion, the best way to measure a poll’s accuracy is to look at its absolute error — i.e., the difference between a poll’s margin and the actual margin of the election (between the top two finishers in the election, not the poll). For example, if a poll gave the Democratic candidate a lead of 2 percentage points, but the Republican won the election by 1 point, that poll had a 3-point error.

https://fivethirtyeight.com/features/2022-election-polling-accuracy

A lot of people here claim that the POLLS predicted a "Red Wave". They didn't. They mostly predicted close races. ANALYSTS took that data and predicted that, in aggregate, the close races would break for the Republicans, because that's what traditionally happens in the first election after a Democrat wins the Presidency.

[ColinC]

1. K/r

I thought they were gonna be way off which would have meant a blue wave. Instead, they were right on target.

[Takket]

2. Agreed

Honestly for months leading up the election the generic ballots were almost dead even. A few points either way. I’ve seen pollsters do awful but this time their numbers said it was close.

Which is why I never understood what the media was looking at (other than their own desires) that continually lead them to predict a massive red wave.

Especially headlines like “democrats are bracing for massive losses” that went on to quote zero Democrats. It read more like “we in the media are hoping you lose because Fascists are great for ratings!!!”

[getagrip_already]

3. they may not have been far off at the end of the cycle, but....

Pollsters like to correct their polls as elections approach so they will be within the margin of error no matter how the elections turn out.

Then they claim the polls were accurate. The annoying fact that were complete garbage for months before the election gets overlooked.

They are playing games with the numbers, which they control and publish.

I call BS on 538 protecting its income here.

[Fiendish Thingy]

4. So, did 538 downgrade the pollsters that flooded the zone with shit in October?

If not, this is nothing more than a bad job of CYA.

Edit: They downgraded one of the worst, if not the worst polling houses, Trafalgar, who flooded the zone with 37 shitty polls from A- to B.

Trafalgar’s polls didn’t just wrongly predict close races (defined by 538 as 3 points or less gap between candidates); IIRC, they had several October polls showing republicans up by 7 points or more, sometimes double digits.

Trafalgar’s polls were unquestioningly included in 538’s aggregate average and projection model, skewing the results significantly towards republicans, thus driving the media narrative. Downgrading them to a B won’t change this- they are still free to flood the zone with shit in 2024, and 538 will be happy to let themselves be manipulated once again.

Trafalgar should have been down graded to an F and excluded from the aggregate averages.

[BannonsLiver]

7. CYA for 538 and gaslighting from the OP

[brooklynite]

10. What was factually incorrect in what I said?

[PortTack]

14. Nailed it!! Another fabulous CYA job by 538!

[Polybius]

5. Looking back, the only Senate race they got wrong was NV

And even that wasn't that wrong, as the Republican was only leading by about 1-2 points. The big mistake was predicting that Republicans would take the Senate. Even if they got Nevada right, that would still be 50-50.

Off-topic but there were no real upsets. How rare is this?

[Johnny2X2X]

6. No Trump

So we've had several elections since Trump came on the scene, and when he's not on the ballot, the polls have been accurate. Trump is the X factor, 2016 and 2020. 2018 and 2022 were accurate.

[lees1975]

8. Except, that those of us who were watching the polls got the impression they were predicting

a red wave. I guess that was just our imagination. "Margin of error" covers a lot of inaccuracy, since they give themselves up to a 10 point swing in some cases. That always makes it easy to pat themselves on the back. Only seven out of the thirty-two polls in their list had an error in favor of Democrats. And that's kind of what we've been saying, that the average error favored Republicans up and down the polls. Clearly, in spite of their explanation, it did.

So we were right if we said the polls favored Republicans.

[iemanja]

9. That's mistaking the media for the polls

The media makes a big deal about minor, even insignificant, polling differences because they want something easy and mindless to talk about. That's not the same as what the pollsters actually release.

For any polling result to be meaningful, it has to be outside the margin of error. The problem is not that those margins are hedges but that people, especially the media talking heads, don't know how to read polls.

[W_HAMILTON]

11. Did you read your own article?

...

Meanwhile, the bottom of the list features quite a few Republican-affiliated pollsters that systematically overestimated the GOP in 2022: RRH Elections, InsiderAdvantage, co/efficient, Moore Information Group. But the most famous of these is probably Trafalgar Group, a pollster whose methods are notoriously opaque but that played a significant role in shaping the ultimately untrue narrative that a “red wave” was building with its 37 (!) qualifying pre-election polls.

[brooklynite]

12. Yes I did

Most professional analysts saw Trafalger for what it was and disregarded their polling.

[Sky Jewels]

13. But 538 used Trafalgar in its aggregate, didn't it?