Bad Stats Strike Again

I have said before, I'm saying now, and I will no doubt say again: if we must have a core curriculum in university, then it should absolutely include a basic reasoning and Statistics requirement. Via a certain individual in the comments section over at Samizdata who shouldn't think without supervision, I came across this gem.

Apparently someone thinks Dave Cameron has a chance in hell of winning the UK election. (Actually, while I doubt he will win the next one, I do think the tide is turning back to the Tories. But it might take a Prime Minister Brown for a year or so first to scare enough people off of voting Labour.) So the marijuana prohibition lobby is out in force peddling their normal half-baked (pun intended) correlation arguments. This one goes something like this: 8 in 10 people diagnosed with "serious mental illness" are "heavy cannibis users."

Now, leaving aside the obvious definitional leeway problems with phrases like "serious mental illness" and "heavy cannibis user" (just what kind of mental illnesses qualify? And how heavy does usage have to be?), this is a great illustration of the cum hoc ergo propter hoc logical fallacy - or, in statistical terms, "correlation does not imply causation."

I think this is the fallacy that separates the mice from the men in the thinking world - because it's so compelling to a naive viewer. After all, correlation does come with causation. In approximately all cases where one thing causes another, the two things will be correlated, so it's any easy mistake to make for newbies to the world of statistics. But anyone who has done any ammount at all of serious thinking about a subject will quickly disabuse himself of this notion. After all, any number of things are correlated that have no causal relationship. For example, a certain amount of proximity is required in order to shoot someone. In 100% of all gunshot cases, the individual doing the shooting must be within a normal line-of-sight of his victim. Does this mean that being near someone and having a gun causes you to shoot them? Obviously, it doesn't. And yet, proximity and gun homicide are heavily correlated. A naive thinker can be forgiven for making this kind of mistake - but public policy "experts" should know better.

Just because 8 in 10 people who have "serious mental illness" (whatever that means) are also "heavy cannibis users" (whatever that means) doesn't even come close to establishing that cannibis causes mental illness. It might just as easily imply any of the following things: (a) that ("serious") mental illness causes ("heavy") cannibis use (possibly because cannibis stabilizes such people?) (b) that there is a third variable involved (for example, some genetic disposition that causes people to use ("heavy" amounts of) recreational drugs also causes the illnesses in question) (c) that it's just a giant coincidence (for example, perhaps people with ("serious") mental illness tend to congregate together, and marijuana happens to be popular in these circles this year). In order to establish that marijuana use actually causes mental illness, obviously some kind of experiment would need to be done, or else a lot of other variables would need to be controlled for.

Here's a cool quote from the article:

Mary Brett, the researcher who prepared the study, said she was angered to hear Tony Blair's remark last year that downgrading cannabis was justifiable because 'it was worth seeing what happened.' She said: 'Was this just some huge experiment conducted primarily on our vulnerable young people? How many of them would, prior to down-classification, ever have been tempted to try the drug? And how many now find themselves with a psychiatric problem, perhaps for life?'

The answer is probably "roughly the same ammount." It is well known that government policy on marijuana has no effect on the usage rate. And as noted above, she hasn't even come close to establishing that marijuana usage causes mental illness of any kind (let alone "serious" mental illness).

I suspect she knows this. What frustrates me is that people can get away with this because the public doesn't know enough about statistics. If people in general were aware of this kind of trick and actively looked for it, cretins like Miss Mary Brett would be embarrassed to go in public and say the kinds of things they say. So, if we're going to cram kids heads with feel-good classes on diversity and intellectual hogwash like deconstructionist literary criticism, wouldn't it be OK to make them learn something useful like statistics? Of course, I have no evidence that lack of core curriculum education in stats causes statistical ignorance (though they are certainly correlated). So let's do the experiment and find out!