Alex, I'll take "Pseudoscience" for $8

Via Samizdata - the following quote from an interesting post by Hu McColloch:

The whole difference between statistics and astrology is supposed to be that statisticians make statements of statistical significance to determine how likely or unlikely it is that an observed outcome could have happened chance, while astrologers are satisfied with merely anecdotal confirmation of their hypotheses.

No complaints about the text - it's absolutely right, of course. And I should clarify that despite the direction the discussion about it on Samizdata has taken, the original quote was for the purpose of putting some perspective on Statistics, not so much to slam Astrology (no doubt the author does think little of Astrology, but expressing that sentiment wasn't his primary point). That all said, I think it raises - quite unintentionally - some interesting lines of thought on Astrology, as well as the point of statistical significance tests in general.

In particular, two questions strike me as worth asking here. First is the obvious: can Astrology even be subjected to statistical significance tests at all? Second - even if someone managed to devise a significance test for it and it passed, would scientists start to take it seriously?

My strong guess is the answer to both questions is "no," and that says some things about science.

Let's start with the first. In order to run a statistical significance test, you have to have some quantifiable data, and I just don't see what that would be in the case of Astrology. How do you quantify a thing like "Aries is aggressive?" Simply put, you can't - not really - since what we're talking about here, presumably, isn't so much measurable behavior as an emotional tendency. Indeed, the way I understand it, what few tests have been done of the "scientific validity" of Astrology have resided in exactly this ambiguity: they tend to show patterns that are greater than chance, but not statistically significant (or at best only barely so, usually with mitigating explanations), leaving believers and skeptics alike happy. The skeptics are happy because the lack of statistical significance confirms that there is no evidential basis for believing in Astrology's claims. The believers are happy because this affords them some weak confirmation of their beliefs, the deficit of which can be explained by the "inherent complexities" of Astrology or whatever (believers in Astrology are fond of pointing out, for example, that there's a lot more to it than just Sun Sign - all kinds of other factors play a role). I'm thinking in particular here about the Mars Effect, of course, but see Hartmann, Reuter and Nyborg 2006 for a review of past research that casts a similar characterization. In short, trying to confirm or refute Astrology from a purely statistical perspective seems hopeless as the parameters of the playing field can't reliably be established. Put simply, the reason that Astrology is pseudoscience isn't so much because attempts to validate its claims have failed, but because the claims it makes are not quantifiable to begin with, at least not in any obvious or easily-agreed-upon way. Which is to say, it is pseudoscience because it doesn't make scientific claims.

Now the second. Even imagining that some sort of test could be devised that everyone would agree on, and this test managed to pass significance, I still don't believe it would help Astrology's scientific reputation much. The reason for this is that there isn't any plausible candidate for what its mechanism would be. How is it possible that the positions of the sun and the planets have any effect on personality? It's hugely implausible, in fact. I suppose an intelligent believer in Astrology would claim that the mechanism isn't the planets themselves - that this is a proxy for something else, something currently mysterious but eventually discoverable. Fine - but Astrology still isn't science until someone has a good theory what that might be, and to date no one that I'm aware of does.

OK - so Astrology is pseudoscience. What a thoroughly uncontroversial thing to say, right? Well, right - except that characteristics I outlined above that make it pseudoscience seem to me to apply equally well to other academic disciplines that are taken more seriously.

One, as I was reminded in Sociolinguistics class yesterday, is Gender Studies. The discussion was on ostensibly "sexist" language and what, if anything, to do about it. We've all heard the drill, right? If the universal pronoun "he" in English is phonetically identical to the masculine pronoun, doesn't this damage little girls by making them, for instance, think that they can never be doctors because they aren't used to hearing the word "she" in connection with that profession? At least, this was the case the instructor (who advocates for the elimination of sexist language) put to us in favor of its elimination. (I should add that she is aware that overt, presriptivist changes are unlikely to be effective and instead advocates slow, voluntary changes - when possible by adopting forms already present in the language.) This is a common opinion among Gender "researchers," and they even cite rigorous evidence that there is a garden path effect associated with supposedly "neutral" masculine forms. For an example of a supposed garden path effect, consider the phrase "one in two men are women." On first pass, this is difficult to parse because "men" and "women" are opposite categories. But of course, once you throw out the masculine reading of men and reparse with the universal reading (meaning "people"), you get the point. One in two people are women, and we're using an archaic term for "people" to mislead the audience into thinking that this is a general problem with the so-called "universal masculine" forms.

In fact, the other example we were given was even more disingenuous. (The source is apparently the Handbook of Nonsexist Writing.)

The average American needs the small routines of getting ready for work. As he shaves or blow-dries his hair or pulls on his panty hose, he is easing himself by small stages into the demands of the day.

It's easy to see what the mechanism is in both cases. In the first case, universal "men" has fallen into disuse, so the universal reading is jarring to our modern ears. As little as 40 years ago, it probaby wouldn't have been. So this is a case of revisionist language prescriptivists using one of their own successes at disseminating a language change to make the case for all the others they want. The sociological correlate of asserting the consequent, really. The mechanism in the other case is that they're taking a clash between "average American" and "panty hose" and pretending it's a clash with "he" and "panty hose." Since "average Americans" do not wear panty hose (even in the female population, panty hose are hardly a universal form of dress, and it shouldn't need pointing out that the "average American" is male as well as female), it is just as plausibly this antecedent that is causing the garden path effect.

That's a bit of a digression - but it leads us to the point, which is that the hypothesis that so-called "masculine universal" forms serve to dampen the career ambitions of young females strikes me as every bit as pseudoscientific, by the outlined criteria, as Astrology is. First, it's not clear how we could agree on a quantifiable test of this effect, even if we found it plausible. It would simply be impossible to control for all the relevant variables and establish that language use itself was playing any kind of quantifiable role here, let alone a statistically significant one. And indeed, even if we could agree on and run such a test and it did somehow manage to meet some objective criterion of significance, does anyone honestly believe this would eliminate the "problem?" Language use is largely creative and spontaneous. The great lie of Nineteen Eighty-Four wasn't that government would try to control language use (it would, and does), but that it would work so well. Deprive people of one word for something, and they simply invent another. Any attempt to de-gender language would fail in the face of real sexist bias. The only reason it works at all is because people are generally sincere in their desire to treat women and men equally to the extent economics allows. Second, what would the mechanism even be? The idea that repeated use of the word "he" as a referent for doctors somehow discourages women from applying to medical school sounds like so much voodoo to me. I'm just not sure how it would work, really - given that other professions that were presumably reserved for men in times past (secretary, steward, teacher) don't seem to have suffered much from the "he-effect." Likewise, has anyone even bothered to check if there is more gender equality in countries that speak languages where there are no pronominal gender distinctions? Now you see what I mean about Astrology. There are such countries, of course, but the Gender Studies language nazis have a ready-made excuse. They'll simply say that linguistic gender bias in these languages takes other forms that need investigation. My point is that this is a hypothesis that is taken completely seriously by the academic community, and yet it's on no better experimental footing than Astrology. Actually, if we're honest, it's probably even on worse footing.

Another one that strikes me is certain kinds of Criminal Psychology - in particular criminal profiling. I don't have as much experience here - but it seems to me that this is a branch of science that would pass the pseudoscience test as well. The way I understand it, criminal profilers are given case studies of captured criminals, and they extrapolate from these the likely behaviors of current, at-large criminals. Some practical uses to which it's put, for example, involve the police subtly altering the suspect's environment to see if he behaves in the way that the profile would predict - presumably increasing their suspicion if he does. But in any event, it's hard to see how any criminal profiling "findings" could be subjected to rigorous statistical tests, given the huge numbers of variables involved in determining human behavior as well as the general lack of a control group (it may, for example, happen to be the case that a large number of serial killers who cut off the fingers of their victims are deathly afraid of flying, but there are few serial killers (fewer still who like fingers) and tons of people scared of airplanes). Likewise, the mechanisms are far from clear. Even if we established something akin to the toy correlation I just invented for the purposes of illustration, what mechanism could possibly be marshalled to explain it? Whatever it is, it would operate in terms similar in their vaguery to the terms in which astrologers speak.

The overall point is that I think the lines between what's "science" and what's not are not really all that clear - especially not in the minds of the general public, but not really in the minds of a lot of researchers either. The bias against Astrology is a prejudice that isn't applied evenly. This isn't, just to be clear, an argument for opening Astrology departments in state universities - far from it! I like Astrology just where it is - in the "heavy reading" section of the supermarket checkout lane. I just find it interesting which examples people choose when they're deriding someone for their lack of scientific rigor - the context of the quote that inspired this. I also think it's interesting that there's an underappreciation of the applicable domains of science and pseudoscience. The example I'm thinking of here is herbal remedies - and it's on my mind because I recently took one - echinacea for preventing a cold I felt developing. The link includes references to reviews of medical literature showing both that echinacea is and isn't effective at preventing developing colds. But so what? My own experiences with it have been positive enough that I was happy to pay $8 for it at a local natural foods store. This particular case adds to my conviction - since I suffered from only light symptoms, and the person from whom I probably caught the cold was in bed for nearly a week. Scientific? Not hardly - but my body my rights, right? The sermon for the libertarian choir being, of course, that too much worship of scientific orthodoxy without understanding the proper place of science is a danger. I'll admit to believing enough in Astrology that I'm generally curious, when someone puts a sufficiently detailed horoscope in front of me, to read what it says. I don't believe enough to pay anyone money for their astrological services, or even to take the time to read a full book on it. Point being, undue equation of "science" with "truth" is ... well, undue. Science imposes standards of rigor that are inappropriate for making the daily decisions that are our perrogative. I have no problem with individual people entertaining the idea that "masculine universal" pronouns kept them from stellar academic success. Hell, people come up with sillier explanations all the time. What I don't want is this kind of superstition being published in respected academic journals, nor being the basis for public policy. And I have no problem with the medical establishment not believing in the immune-system boosting properties of echniacea. They would hardly be doing their jobs if they did. My only concern is that it not be banned as "quack medicine" (even though it technically is), since in my own life I've found it worth the money, and I really do believe that's my decision to make.

The quote that started this post comes from a criticism of a paper on climatology - specifically a review of the proxy data that gave rise to the infamous hockey stick graph showing dramatic warming of the planet in the 20th century (though, of course, it didn't really - hence the controversy). This is a sterling example of a field where people need to get their ducks in a row about what's science and what isn't. That's because people are proposing "remedies" costing in the tens of trillions to a problem no one has satisfactorily established is really all that dangerous - certainly not to the extent a lot of the hype would have you believe. Since it's public policy we're talking about here, I think we definitely do have the right to demand that the "science" in this case really be science - and not Astrology. Or Gender Studies.