Disturbing review of the Cook et al. (2013) 97% paper

[LouisvilleDem]

John Cook's 2013 paper "Quantifying the Consensus on Anthropogenic Global Warming in the Scientific Literature" was certainly one of the most talked about papers in 2013. I've read a number of criticisms of the paper from various places and always dismissed them for being vague and biased. This review is different. It is an extremely well documented (and long) description of the numerous flaws present in the paper--flaws that say more about the integrity of the lead author (Cook) than anything else.

I found this paragraph near the end to ask some very pertinent questions:

Those climate scientists who defended this garbage upset me the most. What are you doing? On what planet would this kind of study be valid or clean? Are you unfamiliar with the nature of human bias? Is this about environmentalism, about being an environmentalist? Do you think being a staunch leftist or environmentalist is the default rational position, or isomorphic with being pro-science? Do you think that environmentalism and other leftist commitments are simply a set of descriptive facts, instead of an optional ideological framework and set of values? Do you understand the difference between 1) descriptive facts, and 2) values and ideological tenets? I'm trying to understand how you came to defend a study based on the divinations of lay political activists interpreting scientific abstracts. Those scientists who endorsed this study are obligated openly and loudly retract their endorsement, unless you think you can overcome the points raised here and elsewhere. I really want to know what the hell you were thinking. We can't be this sloppy and biased in our read of studies just because they serve our political aims. The publication and promotion of a study this invalid and fraudulent will likely impact the future reception of valid studies of the climate science consensus. You might say that we should've hushed this up for that reason, that I should've remained silent, but that just takes us down another road with an interesting outcome.

http://www.joseduarte.com/blog/cooking-stove-use-housing-associations-white-males-and-the-97

[NYC_SKP]

1. Whether or not Anthropogenic Global Warming is reality, we need to fucking stop.

We need to stop our rape of the environment, reduce by 99% our impact on the environment, and we need, probably, to reduce our population to the numbers suggested on the Georgia Guidestones.

The other advice found thereon is also, IMO, good advice:

A message consisting of a set of ten guidelines or principles is engraved on the Georgia Guidestones in eight different languages, one language on each face of the four large upright stones. Moving clockwise around the structure from due north, these languages are: English, Spanish, Swahili, Hindi, Hebrew, Arabic, Chinese, and Russian.

Maintain humanity under 500,000,000 in perpetual balance with nature.
Guide reproduction wisely — improving fitness and diversity.
Unite humanity with a living new language.
Rule passion — faith — tradition — and all things with tempered reason.
Protect people and nations with fair laws and just courts.
Let all nations rule internally resolving external disputes in a world court.
Avoid petty laws and useless officials.
Balance personal rights with social duties.
Prize truth — beauty — love — seeking harmony with the infinite.
Be not a cancer on the earth — Leave room for nature — Leave room for nature.

http://en.wikipedia.org/wiki/Georgia_Guidestones

[dpibel]

2. Gosh! Thanks for you're sincere concern!1!!1

Last edited Thu Sep 11, 2014, 01:30 AM - Edit history (1)

I'm sure I--as do you--want to get my climate science analysis from a fellow who is "a PhD candidate in Social Psychology at Arizona State University, a few weeks counting down to my defense."

Because there's hardly nobody better equipped to analyze climate science than a social psychologist.

Also, it is always really good on a website like this one to rely on a person who says, "Relatedly, my debunkings of invalid and leftist-biased social science and rigged climate consensus studies should not imply that I'm a conservative (or an AGW skeptic.)"

Because here on DU, we sure do hate us some invalid and leftist-biased social science.

Also, we pretty much take as gospel what comes from someone who says "I'm not a conservative or a Republican (not that there's anything wrong with that.) I'm a secular libertarian."

You could say, if you were so inclined, that this is an ad hominem. Maybe so. But what I read of this screed (and it was enough to make me wish I hadn't) made me suspect that, if I followed all the links, and read all the articles, I'd discover that, as a matter of fact, Jose, they were legitimate inclusions.

But, again, thanks so much for being so concerned as to make everyone aware that Jose Duarte, PhC, believes he's onto something.

I'll sure be more skeptical about climate change now!11!!!!

[LouisvilleDem]

3. Interesting Reply

You could say, if you were so inclined, that this is an ad hominem. Maybe so.

Maybe? There is no 'maybe' about your reply being an ad hominem. That is all it is, pure and simple. You've said absolutely nothing substantive and completely failed to address any of the issues the author raised with the paper. It is also clear that you couldn't be bothered to read more than the first few paragraphs, because illegitimate inclusions was only the first of several problems the author raised with the paper. I bet you didn't even get to the part where he shows how Cook blatantly lied when he said "Each abstract was categorized by two independent, anonymized raters." Or that he also lied in claiming that the reviewers were blind to the authorship of the abstracts. Do these things matter to you, or do you believe that lying about your methodology is acceptable, so long as your results are "right"?

[FBaggins]

4. You shouldn't be surprised. Look up "concern troll"

Last edited Thu Sep 11, 2014, 06:36 PM - Edit history (1)

You aren't playing the part well. You're supposed to wait multiple replies before you shift from "I'm having trouble refuting this... please help!" to active defense of the piece.

[LouisvilleDem]

5. Reading comprehension problem?

I never said I was "having trouble refuting this". I read it through several times and checked up on a few of the claims before coming to the conclusion that the points it made were valid. I didn't post it looking for help in refuting it, I posted it in the hope that people here would wake up to the fact that the politicization of the climate change debate has resulted in some people abandoning science altogether--which is a shame given that the science is on our side.

[caraher]

6. Cook's paper doesn't bear on the reality of human-caused climate change

I only skimmed Duarte's critique but if it's accurate - and I haven't run across any convincing rebuttals through casual Googling that rise much above ad hominem attacks - it means we shouldn't point to the Cook paper as proof of the consensus.

Which is fine by me - IPCC is a very conservative (be design) organization and it's clear enough what their consensus view is.

It's worth noting that Duarte is not contesting either the reality of AGW or the existence of a scientific consensus; rather, he is attacking the Cook paper. And for that, he is probably better qualified as a psychology doctoral student than he would be were he a climate scientist. Duarte offers no climate science, so it misses the point to say it's foolish to take climate science from a psychologist.

I took Cook to be doing essentially an updated version of Oreskes' work. The only "fact" imperiled, beyond the reputations of the people involved, is backing for the specific "97% consensus." I don't think that is going to make a decisive difference for anyone whether the "real number" (if such even truly exists) is 95%, 80%, or 99.9%.

But it's just one more thing to cope with when playing "skeptic" whack-a-mole.

[LouisvilleDem]

7. Agreed

I'm glad somebody else around here gets it.

[NickB79]

8. Then do what every other scientific discipline does, and do a replicate study

Design one that avoids what is considered sloppy or fraudulent in the Cook study, run it, and see what results you get. That's how science works.

HOWEVER......

The most telling thing I've seen, in all the critiques of the Cook paper, is how, despite all the supposed flaws in methodology, it still pretty much jibs with previous papers out there that have studied the scientific consensus on AGW. Nowhere have I read of a paper published in the past decade that found ANY significant doubt within the scientific community that global warming isn't currently being driven by man. Are all the authors who've done such studies, often using widely differing methodologies, wrong? I think not.

[Old Crow]

9. On a cursory review, I find the Duarte article untrustworthy.

I've spent a lot of time online bebunking "skeptic" articles. It can take several hours--or more-- to do a thorough debunking and, frankly, that's what the skeptics count on: they'll quickly slap together an agglomeration of semi-valid statements that are interwoven with falsehoods and distortions and publish it, knowing that to debunk the piece in a conscientious manner will take significant effort.

I don't happen to have twelve hours to devote to this piece of Jose Duarte's that LouisvilleDem has presented. I do, however, have one hour, and I've already used 30 minutes of that time to take a quick look under the hood of the piece and see if anything's amiss.

In just thirty minutes, I've found enough to cause me to suspect the entire piece isn't trustworthy.

One of Duarte's chief complaints--he starts his article with it--is that articles that have nothing to do with climate science were included in Cook's paper. He provides a long list of such papers to prove his point. The first article, for example, is titled "Biomass fuel use, burning technique and reasons for the denial of improved cooking stoves by Forest User Groups of Rema-Kalenga Wildlife Sanctuary, Bangladesh." Duarte states that he was "blindsided by the inclusion of non-climate papers," and, yeah, on the face of it, including an article on stoves, of all things, in a survey of papers on climate science seems odd.

Duarte himself, in an annotation, describes the article: "This is a survey of the public's stove choices in Bangladesh," Duarte writes, "and discusses their value as status symbols, defects in the improved stoves, the relative popularity of cow dung, wood, and leaves as fuel, etc."

Wow. Cow dung. I'm starting to wonder just what Cook et al. were thinking.

Fortunately, all of the data the Cook study uses are available for anyone to examine online. Naturally, the first thing I did was to confirm that the abstract was included in the study. And perusing the long list of 12,464 abstract--yes, indeed, it was. No problem there.

The second thing I did was to look up the actual paper and read the abstract.

What's this? In the very first sentence of the abstract, the paper's significance from a global warming perspective is made perfectly clear: "Use of biomass fuel in traditional cooking stoves (TCS) is a long-established practice that has incomplete combustion and generates substances with global warming potential (GWP)." Well, this is odd. Why didn't Duarte mention this? Why would he go on about cow dung, of all things, and make no mention of the fact that the authors' chief concern is that choices in stoves can result in different amounts of greenhouse gas emissions?

In Duarte's description of the article, he writes: "They mention climate somewhere in the abstract, or perhaps the word denial in the title sealed their fate." This statement of Duarte's is even stranger. "They mention climate somewhere in the abstract" (my emphasis)? Well, yeah, if by somewhere you mean the first sentence.

The article's abstract continues: "Improved cooking stoves (ICS) have been developed worldwide as an alternative household fuel burning device, as well as a climate change mitigation." The abstract then goes on to explain how the paper is assessing the adoption rates of improved cooking stoves among the Bangladesh population and the fuel choices of the users. It became pretty clear to me that far from being a farcical discussion of cow dung, the paper was doing real science related to global warming since the subject under consideration has an impact on not only the generation of greenhouse gases but also the cutting down of forest for the purposes of fueling cookstoves.

Let's review: This is the very first article that Duarte provides as an example of "non-climate" articles that "blindsided" him. In a manuever that I can only describe as deceitful, he mischaracterizes the abstract and gives his readers the impression that any connection of the article to global warming is obscure to the point of being non-existent.

It took me less than thirty minutes to research this and quickly uncover a bias which, to my mind, makes the entire Duarte article suspect.

So, LouisvilleDem, I've got to ask: Before quoting extensively from this article, linking to it, and promoting it on here, why didn't you do at least as much checking on it as I did?

[NickB79]

10. BRAVO!

[Old Crow]

11. Thanks!

Unfortunately, the original post was already a few days old by the time I got to it and it's looking like most members of the group have moved on. Oh well. Better late than never, right?

[GliderGuider]

13. A excellent and valiant effort! It appears not to have helped, unfortunately.

"A man hears what he wants to hear and disregards the rest."

[LouisvilleDem]

14. A man hears what he wants to hear and disregards the rest.

I completely agree.

[GliderGuider]

15. The technical name for what Paul Simon described in that quote is "confirmation bias"

Confirmation bias

Confirmation bias, also called myside bias, is the tendency to search for or interpret information in a way that confirms one's beliefs or hypotheses. People display this bias when they gather or remember information selectively, or when they interpret it in a biased way. The effect is stronger for emotionally charged issues and for deeply entrenched beliefs. People also tend to interpret ambiguous evidence as supporting their existing position. Biased search, interpretation and memory have been invoked to explain attitude polarization (when a disagreement becomes more extreme even though the different parties are exposed to the same evidence), belief perseverance (when beliefs persist after the evidence for them is shown to be false), the irrational primacy effect (a greater reliance on information encountered early in a series) and illusory correlation (when people falsely perceive an association between two events or situations).

It's related to a phenomenon called motivated reasoning:

The processes of motivated reasoning are a type of inferred justification strategy which is used to mitigate cognitive dissonance. When people form and cling to false beliefs despite overwhelming evidence, the phenomenon is labeled "motivated reasoning". In other words, "rather than search rationally for information that either confirms or disconfirms a particular belief, people actually seek out information that confirms what they already believe."[2] This is "a form of implicit emotion regulation in which the brain converges on judgments that minimize negative and maximize positive affect states associated with threat to or attainment of motives."

Both are psychological strategies used to cope with cognitive dissonance: "the mental stress or discomfort experienced by an individual who holds two or more contradictory beliefs, ideas, or values at the same time, or is confronted by new information that conflicts with existing beliefs, ideas, or values."

Lest you think I'm dumping on you personally here, this applies to everyone on the planet. These strategies appear to be an intrinsic part of human nature, probably the result of a long series of psychological adaptations stretching back through tens or hundreds of thousands of years of evolution. Consequently, I am as prone to them as anyone else.

The best that we can do to combat their blinkering effects is first to recognize their operation in the views we form, then to expose ourselves deliberately to alternative and contradictory viewpoints, and above all to practice accepting the mental discomfort of the resulting cognitive dissonance. Which explains why you're not on my Ignore list.

[LouisvilleDem]

16. Yup, we all suffer from it

The funny thing about this piece is that the author is attacking the paper, not climate science. He is simply saying that if we want to continue to claim that we have science on our side, we have to call out papers and authors that fail to follow basic scientific practices. Part of the problem with the paper is the authors failed to use even the most basic precautions to prevent confirmation bias from influencing the outcome of the study.

[LouisvilleDem]

12. Thank you

I appreciate the fact that you took the time to read the material from the link.

I would disagree with your assessment that the paper on cooking stoves was properly categorized as being about climate change. Personally I think the paper on cooking stoves is best viewed as a paper on cooking stoves, not climate change. However, I can understand why you may feel differently. The most compelling criticisms that Duarte makes of the paper however, do not concern whether or not certain papers that were included in the study should have been or not.

Two people can obviously disagree about whether or not a paper on cooking stoves should be included, especially given that the selection criteria does not explicitly define what is meant by a paper being 'about' climate change. However, what is impossible to dispute is that the methodology explicitly defined by the Cook paper was not followed. Specifically:

1) The methodology says that papers were selected that contained the phrases "global warming" and "global climate change". However, a paper by skeptic Roy Spencer entitled "How serious is the global warming threat?" was excluded from the ratings. Also excluded from the survey were every single paper written after 1997 by Richard Lindzen.

2) The methodology says that papers on "Social science, education, research about people's views on climate in the papers" would be excluded as 'Not Climate Related'. They were not, and Duate lists several examples.

3) The methodology says that "Each abstract was categorized by two independent, anonymized raters." They were not, as the discovery of a forum used by the raters contains numerous posts showing how they discussed papers amongst themselves, performed google searches to determine the names of the authors, and made plans on how they would market the results of the paper, even before those results were known.

I understand that you don't have sufficient time to review all of the criticisms made by Duarte. I certainly don't have time to review everything either, which is why I didn't respond to your post for several days. However, I think it is unfair to dismiss the entire piece after looking at only one paper Duarte believed was improperly included. He lists a total of 19 papers that he believes were included improperly, and notes that none of the 50 papers Lindzen wrote after 1997 were included.

You asked me why I didn't do as much checking as you did before posting. Since you only looked at one paper of the 25 mentioned, I would ask you the same. Moreover, when one considers that improper inclusions was only one of numerous other criticisms Duarte made, it certainly looks as if you didn't read more than the first four paragraphs of a piece that is dozens of paragraphs long. Given this, I'd say that it is clear that I have spent quite a bit more time researching the piece than you.

If you really want to convince me though, perhaps you could explain how a paper by one of the world's best known skeptical climate scientists (Richard Lindzen) entitled "Do deep ocean temperature records verify models?" gets excluded while a paper entitled "Space technologies for the building sector" by a German aerospace engineer gets included. Take your time, because I know you are busy.

Thanks again for taking the time to read the material at the link.

[NickB79]

17. Skeptical Science: Is there a scientific consensus on global warming?

http://skepticalscience.com/global-warming-scientific-consensus.htm

Several studies have confirmed that “...the debate on the authenticity of global warming and the role played by human activity is largely nonexistent among those who understand the nuances and scientific basis of long-term climate processes”. (Doran 2009). In other words, more than 97% of scientists working in the disciplines contributing to studies of our climate, accept that climate change is almost certainly being caused by human activities.

The Cook paper is just one of many that have attempted to quantify the scientific consensus.

Even if it were found to be complete garbage, it wouldn't change a thing, because multiple other papers by other researchers have come to the same conclusion independently of one another.

[LouisvilleDem]

18. Then you should have no problem denouncing it

I agree, the fact that the study is horribly flawed means nothing in terms of quantifying the scientific consensus. However, bad science should be pointed out regardless of which 'side' produced it.