How to Generate Bogus Conclusions (E-Cig Study Edition)

[DanTex]

114. Yes, the population of e-cig smokers is self-selected.

Which means that it is probably different in certain ways than the population of non-e-cig smokers. The thing is, epidemiologists know about this problem. You're not the first person to think of this.

In fact, this happens all the time in epidemiological studies. Even the studies that originally found that smoking caused cancer had the same problem. The population of smokers, like the population of e-smokers in this study, is self-selected. And yet epidemiologists are able to compare them to the population of non-smokers and draw statistically sound conclusions about rates of lung cancer. How can that be?

The answer is something called multivariate regression. In addition to just collecting the data of whether or not someone smokes/e-smokes, they also collect a whole bunch of other variables that might affect whether they get cancer/quit smoking. If you click on that link to the first page of the study, you will see some of the other variables they looked at. You'll see that intent to quit is there. In fact, not only do they distinguish between people trying to quit and not trying to quit, they in fact have four different levels of trying to quit.

They also collect data about gender, about age, about education, race, and how much a person smokes. Because, for example, it might be possible that e-cig smokers are more (or less) educated than the smokers who didn't try e-cigs, and more educated people are more likely to quit smoking period, for other reasons that have nothing to do with e-cigs. If this were true, then a correlation would show up between e-smoking and quitting, but it would be spurious, and education would be a confounding variable.

Anyway, they take all these potential confounding variables, and they build a statistical model that isolates the effect of each one of them independent of the others. So, for example, if e-cig smokers were more likely to quit, but the entire effect was due to confounding based on education level, they would be able to see that in the data.

Since the paper is behind a paywall, we don't know the details of the multivariate analysis and what controlling they actually did. But it's a pretty good bet that they did at least a reasonable job controlling for confounding variables, because otherwise it's hard to imagine it would have gotten into a decent journal.