Criticism of Freeman and USCountVotes

[davidgmills]

The author, who is banned from DU, asked for me to post his article. In the interest of fairness on the issue, and for critical review, I thought I should.



http://www.stonescryout.org/archives/2005/03/critical_review.html

[Mairead]

1. Let's see, he starts out with a personal attack

and then assumes his conclusions. Uh-huh. Good work, kid.

I don't know whether to hope his advisor is or isn't reading the stuff he's putting out. :( I feel bad for him, but also irritated. His politics seem to deny him insight into what he's doing, and that's a serious problem for anyone in science to have.

[dzika]

2. Considering the source and the timing....

Rick Brady isn't exactly a bastion of non-biased anything.

Just as the the neocon staffed "American Center for Voting Rights" pops up with a report only few days before the Ohio vote congressional hearings on Monday, Rick Brady pops up with his anti-Freeman report one day before the hearing.

I'm sure it's just a coincidence... not.

[FreepFryer]

3. *yawn* faith does not balance truth in the interest of fairness. (n/t)

[garybeck]

4. since when is writing a book an indicator that you're wrong?

it is a repeated tactic of the republicans. every time someone makes a case against them, they point out that they wrote a book, so they obviously must just be making the whole thing up in order to sell books and make money.

I'm so sick of that argument. Writing a book has nothing to do with credibility. If anything it adds to it. Most people wouldn't have the guts to make something up and put all that time and effort into writing a book about it, knowing it would only discredit them professionally.

I remember in 9th grade debate class, they taught us the basic rules of debate - no personal attacks or name calling. focus on the issues not the person. In every discussion about every issue, it seems the republicans break these basic rules every time. Must be because they don't want to actually talk about the issue.

[FreepFryer]

5. Right you are, Gary. They are unable to debate the issues. (n/t)

[Melissa G]

6. Rick goes after Freeman for not being an expert ?

From the website here are Rick's credentials...

Rick Brady is a planning consultant with a background in land use, housing, and environmental planning. Rick has provided policy consultation services to dozens of public and private sector clients throughout California.

In 2001, Rick attended the Horizon School of Evangelism and served as Assistant Director for the team's month-long practicum to Querétaro, Mexico. Currently, he serves as the worship leader for his home fellowship and sits on the Balboa Little League Board of Directors.

Rick graduated with Highest Distinction from the University of California at San Diego (B.A.) and is currently pursuing a Master's in City Planning (M.C.P.) from San Diego State University. He and his wife live in San Diego with their two young children.

Rick started blogging in September 2004 at the original Stones Cry Out, but now makes his home at www.rickbrady.net.

Here is the websites Vision...

"If they keep quiet, the stones will cry out." Luke 19:40

Stones Cry Out is a group blog produced by five contributors who seek to make the site a daily crossroads (Jeremiah 6:16) of news, analysis, spiritual insight, accountability, humor, and outrage. We present a mix of original content and links to the best of the blogosphere. SCO captures the work of a diverse group of professionals who are committed to being constantly relevant, always reverant, and fearlessly honest.

We believe the role of a blog produced by followers of Christ is to--like Paul at Mars Hill--"reason in the marketplace day by day with those who happen to be there" (Acts 17:17). As we contend for the faith, we seek to bring glory to Jesus Christ (Luke 19:40), and advance His kingdom.

Enough said....

[Name removed]

30. Deleted message

Message removed by moderator. Click here to review the message board rules.

[Mairead]

32. "It seems to me that there is something to this paper"

Yes: the 'something' starts with a personal attack (which, besides being ludicrously hubristic in content, conforms neither to scholastic nor christian practice) and then goes on to assume his conclusions, namely that Mitofsky and the official results are correct.

Once someone has assumed their conclusions, anything that follows will necessarily be self-justification rather than scientific inquiry, so I stopped reading there.

[Helga Scow Stern]

7. Curious to see what "Mystery Pollster" has to say about this....

it will be interesting to see how much his true position is revealed in whether or not he defends a substandard piece of work.

[dzika]

8. Substandard? I surprised at you Ojai...

It has footnotes. So it must be true.

[Helga Scow Stern]

35. Yep, especially if Mystery Pollster backs it up.

But even he might not. I wonder what he'll have to say about the new US Counts Votes report? Nevermind. I think I already know.

[davidgmills]

9. I have argued with Rick for some time now

But I think I should sit this one out.

I realize that if we went to freeper land they would probably ban us, but I don't know that we should react in kind. As liberals, to the extent feasible, I think we should hear all sides. He is genuinely interested in this topic and wants to be heard. That was the reason for my post.

[jkd]

10. I'm with Rick

I basically agree with Rick. Freeman underestimates the NEP exit poll’s design. While the poll deviates toward Kerry, it is not near as significant as Dr. Freeman suggests. Mitofsky still should be concerned about the WPE, however; I certainly hope they're working to correct the poll’s bias.

I have followed the BYU polls for some years. They interviewed ten times as many people in Utah as NEP, and did exit polls in six times as many precincts. Their MOE was 2.6% and Mitofsky was 5%. Of course all polls aren't created equal.

[davidgmills]

11. Did Mitofsky really care about Utah?

Seriously doubt it. Five percent MOE was good enough to predict the most (or nearly the most) lopsided state in the union.

[jkd]

12. No, If the purpose was simply to predict the winner.

You don't have to tell me about the political realities of Utah. I have to deal with it almost every day.

[Mairead]

13. " Freeman underestimates the NEP exit poll’s design"

Think about this for a moment: there is no proof at all, or for that matter even any evidence, that the fudge factor Mitofsky selected is correct. It is a handwaved number, pulled out of thin air. The arguments put by Mitofsky, MP, and Rick all presume without any supporting evidence that the official numbers are correct, and all their 'explanations' are nothing but post-hoc, handwaved attempts to justify that presumption.

That's not science, that's religion!

When you support Rick et al., you support nonsense. Freeman is doing the right thing.

[jkd]

15. " It is a handwaved number, pulled out of thin air.'

Mitofsky published the MOE for the national poll and each state poll before the election. Freeman chose to determine his own MOE, without access to the factors that determined the poll's design. Who "pulled (the number) out of thin air"?

[Mairead]

16. "Freeman chose to determine his own MOE"

There are near-cookbooks for these things, you know. It's not quantum electrodynamics.

Freeman has a PhD in Organisational Psychology: the psychosocial dynamics of groups. One of the requirements for that degree is mastery of statistical methodology, and in his speciality his focus would have been on how to investigate and sample group behavior.

Mitofsky assumes (in public, at least) that the official count is correct, and everything he says follows from that. His speculations about the etiology of those trout-in-the-milk discrepancies are nearly gratuitous. His chief handwave--that GOPers must have responded less often--is contradicted by his own data. How strange that he didn't notice that. Other people, who are either nitwits or think we are nitwits, have suggested that it was because young people and/or women were disproportionately sampled. But the only way some subpopulation can inadvertently be disproportionately sampled is if it's defined by an invisible characteristic. Age and sex membership are not invisible. And so the beat goes on.

[jkd]

17. I'm not defending NEP

I think their poll is inadequate for determining the winner in a close election. They are, however, privy to the data (the precincts polled and the demographic of each). They have the advantage over Freeman. I don't care how masterful Dr. Freeman is in his methodology, without that data, he can only guess about the margin of error for Mitofsky's poll.

And how does one know how good the guess was? Yes, one must compare it to the election results. The evil election results. Just like Mitofsky does.

[Mairead]

19. Of course you're trying to defend NEP! Don't insult me.

The only reason Mitofsky's estimate would differ to Freeman's by more than a trivial amount would be if Mitofsky were trying to factor in his guess about the current amount of election fraud and Freeman was assuming no fraud. It's a near-cookbook process! The 'design' that gets so much hype is not how to get accurate results, but how to do it cheaply.

(edit) Freeman was working from coarse numbers, but they were Mitofsky's coarse numbers, supplied to his clients. There is no reason to think them inaccurate, since Mitofsky's business depends on them being solid and bankable. Now, someone working from coarse data can get a little effect, maybe because the coarse data shows a zero because of rounding but it was really a 4, and so someone who assumes that it's not rounded will make a little error and maybe say it's 19:1 instead of 18:1. But finding that it's 600K:1 is not the result of working with coarse data! (/edit)

And how does one know how good the guess was? Yes, one must compare it to the election results. The evil election results. Just like Mitofsky does.

Anyone who judges the accuracy of B by assuming sin evidentio the accuracy of A is either a crook or a fool.

[jkd]

21. I am not a fan of the NEP

Neither Freeman nor Mitofsky's clients had access to the poll design.

I sent this e-mail to Dr. Freeman ten weeks ago:

"Dr. Freeman,
The Utah polls surveyed 90 precincts and had over 9000 respondents. Mitofsky surveyed 15 precincts with 828 respondents. The BYU poll's primary purpose was to predict the election. NEP was after a multitude of demographic data with their poll.

The BYU students surveyed my own precinct. I had the opportunity to assess the quality of their work. I believe they were competent and extremely motivated. They have been doing these polls for over twenty years.

Mitofsky suggests a red shift of over four percentage points; the BYU polls would not really show much variance. Either NEP did a poor job or they were part of a conspiracy. Perhaps it wasn't such a poor effort; the NEP site gives a + or - 4% MOE for these state exit polls.( Utah's MOE was +/-5%) Maybe we have been led to expect too much from these polls.

If Mitofsky was only attempting a prediction, he could have done much better. He would have required more resources like those expended in the Utah polls."

I am not a fan of the NEP. I thought their poll was inadequate and their interviewers poorly trained. I don't believe Mitofsky is dishonest, however.

[Mairead]

25. You seem a fan of the idea that the official numbers are correct, tho

The Utah polls surveyed 90 precincts and had over 9000 respondents. Mitofsky surveyed 15 precincts with 828 respondents.

Here's that cookbook thing again. There's a threshold number of samples you need to get a certain confidence that your results are correct. As long as you reach that level, you're okay. Sampling more than that number increases both confidence level and also, usually, cost. So most people don't do it.

The fact that the students did 9K samples for Utah and Mitofsky only did 828 (I take your word on the numbers) does not impeach the goodness of Mitofsky's results. Eight hundred, for a low-pop state like Utah, sounds like quite a reasonable number (though I should probably say, as I've said elsewhere, that I'm not myself a methodologist and I can't now find my one grad-school stats text that covers the methodology of clumped sampling, so the number might really be low. But I doubt it, because getting it right is Mitofsky's bread-and-butter.)

And, unless Mitofsky's contract specified that he has no ownership rights at all (something he has not claimed, afaik, nor would I expect that to be true), the controversy over his results should have had him practically demanding at gunpoint that his data and methodology be peer-reviewed, under NDA if need be. I find it significant that he has not only not requested any such review himself, but has completely stonewalled the proactive requests made by qualified peer reviewers. That strongly suggests he has something to hide and might well now be 'sanitising' his data by cooking it.

[jkd]

28. No proof Kerry won

I would be ecstatic to see absolute proof that Kerry won. Let me get that out of the way.

The BYU exit poll verified the official vote in Utah. I believe it was a more accurate poll than the one conducted by NEP. With the NEP Utah poll, one sees the same bias in favor of Kerry that was typical of many other states. It was a higher deviation than average, but certainly not as high as a state like New Hampshire.

I eagerly awaited the recount by Nader of selected precincts in New Hampshire. I contributed to the cause. Again the recount substantiated the official election results. Many had their hopes pinned on that recount.

Ohio, what happened in Ohio? Why does one hear no more cries for recounts on this forum? They were plentiful at one time.

[Mairead]

31. And your point is...?

A p=0.01 confidence (99% confidence) means that, on the average, out of every 100 times one gets a particular result, one of those results will have been false because of chance.

Freeman's analysis says, correctly, that the odds of chance having produced Bush's alleged upset wins in those 3 pivotal states is less than 1 in 600,000. Functionally zero, in other words.

So there was something other than chance at work.

Mitofsky suggested that Dems were unintentionally oversampled. His own data contradicts that. Others have tried to suggest that young people or women were oversampled. But that's outright nonsense: age and sex membership are visible characteristics.

So what was it? Fraud is the leading contender, given all the suppressive tactics that have been documented.

Now, it's possible that there was something else, but nobody has put forward any credible evidence at all for the existence of that hypothetical 'something else'. It's all been handwaving, while Mitofsky sits on his fine-grain data. What's he doing with it? Is he sanitising it by cooking? What legitimate reasons could he have for not releasing it, under NDA if need be? I can't think of a single one.

So no, there's no actual proof that Kerry won. But, like Thoreau's trout, all the evidence points to that conclusion.

[jkd]

33. My points

I’ll boil it down to two points. 1) Freeman understates the MOE because he has no access to Mitofsky’s poll design. 2) When the election results have independent verification (an alternative exit poll or recounts) along with the Mitofsky polls, the election results are justified and the NEP polls are “found wanting".

You should read Rick Brady's review; it addresses many of your concerns in your post #31.

[Mairead]

34. I thought we'd come around again to the starting point

Brady doesn't have access to anything either. The claim that Freeman's numbers are wrong IS UNSUPPORTED! He HAS NO SUPPORT FOR HIS CLAIM!

You, like Brady, are assuming your conclusion.

Since I don't feel like having another little meaningless dance with you, I'll drop out here.

[fasttense]

14. This is the same old disinformation from

the right that has been regurgitated to sound official. I'm not impressed nor convinced by Rick's faulty logic. Since he obviously believes Saint b*sh won fair and square, I was hoping he would address the fact that the most powerful and richest country in the world is unable to design an accurate exit poll. But of course he did not go into that. I suppose an attack on Freeman's credentials is good enough for those looking for a way to believe in the accuracy of 2004 vote count.

[autorank]

18. davidgmills...Wow, I'm so happy the religious right has a DU PR arm.

"Fairness" as in there are two sides to every issue. Neither Freeman nor TIA started out with a conclusion which then lead to a search for facts. The facts of the case plus accepted methods of analysis led them to their conclusion.

:wtf: is going on when "fairness" is presented in the form of a right wing blog at the request of a banned DU user.

I find you post highly offensive.

[davidgmills]

24. I find your attitude very offensive

Perhaps "fairness" was a poor choice of words. I am not trying to defend the guy. I put his article up here for critical evaluation. My post is doing what I hoped it would do -- provoke critical argument pro and con.

Let's leave the religion out of the science, please.

[anaxarchos]

37. Good advice for Rick Brady:

"Let's leave the religion out of the science, please."

[skids]

20. If folks REALLY want to test the NEP results for accuracy.

Here's how: This far after the election, each town or county clerk is likely to have a "drudge list" available, at least in those states where such a thing is done. What this is, is a list of all the people who voted (signed the pollbook) in the last election. They are often used by political parties to prepare mailings. All clerks have to have them, since they have to determine a voter's "active" status.

These lists sometimes include the race and gender of the voter, or can be cross-indexed against a pre-election voter registration list to do so. Almost always, they contain the party affiliation of the voter. It is almost unheard of for them NOT to include the age of the voter.

Using these lists, statewide, state by state, it is possible to come up with an actual count of demographics that is independent of the exit poll. Since gross statistics were posted for North Carolina (not lists, but demographic percentages) I was able to look at these numbers and compare them to the NEP results. That showed the actual error in several NEP categories and shed a lot of light on what went on with raw error and subsequent weighting. From this we can see that the weighting applied did not serve to improve the accuracy of the poll by correcting for sampling. All it did was bring the final vote numbers into line. All the statistics on the MSM exit poll sites are out of whack -- anyone planning an election strategy or trying to figure out "what happened to the youth vote" or "how did latinos break down" based on what the MSM has posted is working with numbers that are quite wrong, FWIW.

Personally, until someone explains to me in great and convincing detail why they use the second moment as their preferred averaging method, I will never entirely trust a statistician's results. However Freeman is right, this needs to be investigated without prejudice, and after seeing the crude "bump-up" method used during the final weighting, I *definitely* don't trust Mitofsky's results.

[dzika]

22. skids,

Can you share the link for the "gross statistics" in NC? Do you know if any other states did this? Thanks!

[skids]

23. See post #5 in this thread here:

http://www.democraticunderground.com/discuss/duboard.php?az=show_topic&forum=203&topic_id=338935#343081

...the "gross stats" I refer to are the "Turnout Statistics" PDF file (turnout11xx02xx2004.pdf) on the page linked there.

[TruthIsAll]

27. The proof is on the Global level: Party_ID and How voted.

As I have shown in my recent posts, the "How Voted in 2000" Demographic is IMPOSSIBLE.

The NEP Final 13660 Exit Poll claims Bush 2000 voters were 43% of total 2004 turnout, or 52.57 mm (.43* 122.26).

This overstates the Bush 2000 vote (50.456 mm) by 2.1 million.

The absolute MAXIMUM Bush 2000 voter turnout was 41.26%, assuming ZERO DEATHS SINCE 2000.

Assuming the U.S. annual death rate of 8.7 per thousand, that means approximately 1.75 million Bush 2000 voters died.

So the NEP overstated the live Bush turnout by 2.1 + 1.75 = 3.85 million.

I'm not done yet.

An unknown percentage of Bush 2000 voters who are still alive stayed home, for whatever reason, in 2004. Let's assume it's 1%.

Now 1% of 50.456 mm =.50 mm.

Adding .50 to 3.85, we get a 4.25 million (9.5%) discrepancy between the impossible NEP Bush turnout and the above calculation.

So if this confirmed NEP demographic is off by 9.5%, how far off are all the others?

[PATRICK]

26. Still boils down to

the call for true vote audit(if it were possible) more than honing both polls and the studies of polls into some form of infallibility.

Otherwise it is an attempt to make the war of variables into some decisive substitute for a recount. Either way the loser loses and the poll is trash, but the main point that- like other exit polls in mpst places and most times- such results call for a very close study of the actual results not a rationalized dismissal of the polls with the ultimate in backward logic.

I am in a dark basement. I hear a hissing noise and am getting wet. Am I really wet? What is the cause? Maybe something is wrong with my perception...but... DON'T TURN THE LIGHTS ON! Let's keep arguing until I drown.

[berniew1]

29. But solid documentation supports Freeman's exit poll results

http://www.democraticunderground.com/discuss/duboard.php?az=view_all&address=203x349702

There is documentation from other means of the swings and also of what caused the swings, not only in Florida as above but for other states as well

http://www.flcv.com/summary.html

[southwood]

36. Rick Brady

At some point, months ago, I got involved in a discussion with Rick Brady at mysterypollster.com. The distinct impression I got was that he was much more interested in his own point of view than in that of others. I was amazed to find out that he was an evangelical interested in statistics (but no particular expert in it), but figured this must be very American. Anyway, he always thought Mitofsky was brilliant and critics were no good. He still does, apparently.
Whatever, what Mitofsky should do, is release the complete data on his exit polls, including the exact precincts he surveyed. Now! Otherwise, what is he hiding?

[althecat]

38. Rick is a town planner.& everybody knows town planners are statistics guns

....

Rick is a planning and development consultant with a background in community,
housing, and environmental planning. Mr. Brady has provided policy consultation
services to dozens of public and private sector clients throughout California.
Mr. Brady graduated with Highest Distinction from the University of California at San
Diego (B.A.) and is currently pursuing a Master��s in City Planning (M.C.P.) from San
Diego State University. He is the author of ��A Marriage of Convenience? Fiscal
Incentives and Residential Development Patterns in California�� published in the journal,
Planning Forum, Volume 10, 2004.