If you are a fan of science news, then odds are you are also intrigued by brain imaging, the technique that produces those colorful pictures of brains “lit up” with activity, showing which regions are behind which behaviors, thoughts and emotions. So maybe you remember these recent hits: which regions of the brain listen to angry voices, which regions are active when women grieve the break-up of a romantic relationship, activity showing the problems cocaine addicts have in responding to rewards, how social rejection increases activity in the same brain regions as physical pain, how men and women show different brain activation patterns when they think about their partner cheating on them sexually (men: regions involved in sexual and aggressive behaviors such as the amygdala and hypothalamus become more active) or emotionally, which brain regions become more active when arachnophobes think about spiders, which regions become active when you first experience intense romantic love . . . the list goes on and on and on. And now a bombshell has fallen on dozens of such studies: according to a team of well-respected scientists, they amount to little more than voodoo science.
The neuroscience blogosphere is crackling with–so far—glee over the upcoming paper, which rips apart an entire field: the use of brain imaging in social neuroscience, which studies how the brain processes/produces/handles the social and emotional aspects of human behavior such as jealousy, grief and altruism. This field includes stuff like the above, and has exploded in the last 10 years as psychiatrists and social psychologists became enamored of fMRI and other brain imaging toys. (My guess: like so many researchers in the social sciences, they have physics envy, and think that the illusory precision and solidity of neuroimaging can give their field some rigor.)
The new paper (to be published in Perspectives on Psychological Science but available here) is called “Voodoo Correlations in Social Neuroscience,” which gives you a pretty good idea of its argument. Basically, the authors noticed that a lot of papers in social neuroscience that use brain imaging were reporting correlations between brain activity and social/emotional behavior or thoughts that looked too good to be true or, even, mathematically possible (kind of like the years of steady investment returns that Bernie Madoff reported). So the scientists, led by Edward Vul of MIT and Harold Pashler of the University of California, San Diego, picked 54 such studies, many of them published in prominent journals such as Science and Nature, and wrote to the authors, basically asking how they managed to get such impressive correlations.
More than half admitted using a statistical strategy that, write Vul and his colleagues, “grossly inflates correlations, while yielding reassuring-looking scattergrams.” Other statistical snafus, they say, “likely created entirely spurious correlations in some cases,” and they call on social neuroscientists who use fMRI to reanalyze their raw data “to correct the scientific record.”
What’s striking about the discredited papers—though in fairness, the skewered authors should be given a chance to defend themselves—is how blithely they tend to (as mutuallyoccluded put it) “vindicate the crudest of stereotypes—that women love shopping because they’re “gatherers”, that girls have different kinds of brains and need to be taught separately, that gay men and straight women read maps similarly.” If you were wondering how, exactly, problematic studies got past the peer review at these top journals, that’s a clue: scientists no less than other mortals love to have their hunches, prejudices and stereotypes validated by empirical evidence. Maybe they didn’t look too critically at studies that did exactly that.