firecat

http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/

Ioannidis...and his team have shown, again and again, and in many different ways, that much of what biomedical researchers conclude in published studies—conclusions that doctors keep in mind when they prescribe antibiotics or blood-pressure medication, or when they advise us to consume more fiber or less meat, or when they recommend surgery for heart disease or back pain—is misleading, exaggerated, and often flat-out wrong. He charges that as much as 90 percent of the published medical information that doctors rely on is flawed....Yet for all his influence, he worries that the field of medical research is so pervasively flawed, and so riddled with conflicts of interest, that it might be chronically resistant to change—or even to publicly admitting that there’s a problem.

The article is infuriating because it uses the same kinds of exaggeration that it's criticizing. ("one large randomized controlled trial even proved that secret prayer by unknown parties can save the lives of heart-surgery patients, while another proved that secret prayer can harm them." Um...I believe that individual scientific studies themselves don't pretend to "prove" anything?)

But I do agree with the point that, due to biases and pressures built into the system, the results of medical research are not reliable.

Crossposts: http://firecat.livejournal.com/692539.html

Flat | Top-Level Comments Only

From:

firecat

I think it's possible to write in an exciting style without being inaccurate or overexaggerating, but I might be wrong...I'm certainly not getting published in The Atlantic. :-)

From:

flarenut

Not that it matters so much, but I think (and of course I would) that the kind of hyping the article does is of a different, mostly less pernicious kind than what it decribes. Between publication bias, bad design, post-hoc cherrypicking and all the other tricks, you can get completely bogus results described in the most sober, apparently objective, thoroughly convincing fashion you could ask for.

I think Ioannidis also misses an important point about how information works. If you confirm some item of common belief, you're not adding (in an information-theoretic sense) much to the world's knowledge. If you disconfirm or show something new, you're adding a lot more, and it's only reasonable that people will pay more attention. Of course, something new or disconfirming is more likely to be wrong, but ideally there would be checks and balances.

And of course the key rule of thumb in all of this: one out of every 20 random sequences will produce a result that is significant at the 95% level.

From:

firecat

Those are really good thoughts.

From:

cleverthylacine

It absolutely is. But there are trends in what does and does not get published. A few years ago I was seeing a lot of articles in magazines about how it was more important to eat well, get enough sleep and exercise some than it was to be under a certain weight. You never see that these days; you see all this scare stuff about how "obese" people are all going to die, even though epidemiologically, people at or below what's considered "normal" weight die sooner than those who are somewhat above it. (Being very large is a risk factor, but I've never seen a single one of those studies risk-adjusted to take into account physician bias, and how doctors assume that everything that is wrong with a fat person is because of their weight, which causes necessary testing to be delayed and drugs to be withheld until problems have become much more serious.)

A few years ago, there was all this freakage about how we should assume people are bipolar first, because antidepressants can make people who are somewhat but not very bipolar go manic. Now they're starting to realise that you really don't want to put anyone on an antipsychotic unless you're absolutely sure they need one, because a manic episode can be fixed, but some of the antipsychotic side-effects are horrible and hard to get rid of. :/

And there are a lot of magazines that pounce all over this stuff. You can count on any woman's fashion magazine (with the exception of those marketed for plus women) to be all over the fat scare/fat hate stuff. "Childhood obesity" sells too.

Edited Date: 12 Nov 2010 10:16 pm (UTC)

From:

firecat

But there are trends in what does and does not get published.

There sure are.

I think my "favorite" was "this generation of children will have shorter lives than their parents, because of obeeeeesity." Especially when it showed up in the Parks & Rec newsletter of the town I live in.