In December, I wrote about the brouhaha that was started when the journal Science published a paper suggesting that specific bacteria could survive by using arsenic in their DNA rather than the phosphorus molecules that all other living things rely upon. Those findings were immediately questioned — in blogs, on Twitter and in the mainstream press — and I saw a turning point in the slow, deliberate way in which scientific debates progress.
But this week, I’m questioning whether the rapid reaction to the arsenic paper was really the start of a trend. A new paper just published questions one of the more intriguing scientific findings of the last decade: the idea that obesity is “socially contagious.” The original research, published by Nicholas Christakis and James H. Fowler in the New England Journal of Medicine in 2007, found that people were hugely more likely to become obese once their friends became obese, a pull so strong that it even impacted friends of friends. But the new paper, published by Russell Lyons in the lesser-known Statistics, Politics, and Policy, holds that the NEJM study has its stats wrong.
I don’t know statistics well enough to wade into the debate, but what is was remarkable is how little debate there is, at least out in the open. The new critique was published 4 years after the original research (in no small part because of difficulty finding a journal to publish the results) and wasn’t aired in any meaningful way ahead of time. The authors of the NEJM paper in question have launched a rebuttal to the critique with a draft journal article of their own, responding in part to a draft of Lyons’ paper. That hasn’t received much attention, either. (Christakis said last year, in an interview for a rare article on the controversy, by Slate, that the subject was best discussed in the peer-reviewed press.)
In some ways, this is the way that science is supposed to work, through thoughtful, slow peer review. But that slow pace no longer serves the end of science. Christakis and Fowler’s original research — right or wrong — has become canon, and it doesn’t seem likely that this late objection to their statistics will change the popular perception of the idea, which has already been cited more than 200 times in the medical literature. A draft of Lyons’ work has been out there for more than a year, but with the exception of a couple of small articles, a few blog posts and a handful of tweets, his paper has been paid little mind. The strictly academic discourse might be all well and good if the original research was a topic of interest only to academic specialists. But it wasn’t: it was front-page news in New York Times. Waiting months or years for objections stifles discussion at the exact moment when we are more in need of that debate.
Contrast the obesity flap to the arsenic paper, where the immediate and robust debate helped immediately elucidate the strengths and flaws of the published paper. By the time the official reactions to the publication were made available by Science last month, 6 months after the initial blowup, they added little to the narrative. The community was already educated, and they had already had a chance to consider all the viewpoints while the issue was still fresh. Not so with obesity and social networks.
I don’t want to suggest that the peer-review process is a dinosaur and should be scrapped in favor of he-said, she-said science-by-blogging. But there is a need for both approaches. Eschewing discussion out in the open diminishes our ability to learn at the moment we are most curious, and it hides serves to hide real and important discussions that the media and public need to hear. In the aftermath of the Lyons paper’s publication, at least one doctor has wondered aloud if journalists should have done a better job with stats and the skepticism four years ago. But that’s unfair to reporters; when the scientific community isn’t willing or able to have a real-time discussion about stats or skepticism, the ability of the media (or communications pros, for that matter) to fill that gap so is made considerably more difficult.