Reply #4: A large problem with the medical/public health research is that it ignores its own standards

[Euromutt]

John Ioannidis has an article published almost five years ago titled "Why Most Published Research Findings Are False" (http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124). This is not a problem, broadly speaking; science is largely a problem of elimination, so a lot of hypotheses that are researched tend not to pan out. Initial research may indicate the possible presence of something significant, which subsequent research then shows to have been a statistical fluke.

The problem with the "epidemiological" approach to research is that it relies pretty much exclusively on retrospective studies, i.e. ones that "look back" by examining data already gathered for other purposes. This type of study has its place, e.g. as case control studies in medicine, as they're a comparatively cheap way of seeing whether there's enough substance to a hypothesis to justify further research. But, in the words of research oncologist and medical science blogger David Gorski (http://www.sciencebasedmedicine.org/?p=2962):

<...> here’s one thing to remember about retrospective studies in general. They often find associations that later turn out not to hold up under study using prospective studies or randomized trials or, alternatively, turn out to be much weaker than the retrospective study showed.

As a result, the findings from any retrospective study should not be accepted as valid until confirmed by prospective studies and/or randomized trials.

Now here's the thing about the "epidemiological" approach to firearms research: the material is all retrospective studies. The entire body of work never goes beyond establishing that there's a hypothesis that merits further research. It's all very cute for someone like Charles Branas to claim his team used "the same approach that epidemiologists have historically used to establish links between such things as smoking and lung cancer," but that conveniently overlooks that a) Doll's retrospective study produced a much stronger association than Branas' (90% of lung cancer patients studied by Doll turned out to be cigarette smokers, as opposed to the 6% of shooting victims studied by Branas who were carrying), and b) Doll's research still had to be validated by subsequent cohort studies. And when it comes to firearms, those validating studies are never done.

Why this reluctance among researchers who take the "epidemiological" approach to do follow-up studies, including of research by others in the same field? The most obvious hypothesis (if anyone can come up with something more plausible, let me know) is that they're quite aware that the associations their retrospective studies have generated will evaporate, or at least be severely weakened, in prospective studies. The associations generated by retrospective studies are the best (hell, the only) evidence they have to support their agenda, so they can't afford to jeopardize it. But avoiding evidence that would undermine your hypothesis reduces your work to pseudoscience, and it's hard to escape the impression that is what the "epidemiological" approach in firearms research is.