Peter Kramer (author of
Listening to Prozac) weighs in on the antidepressant re-analysis. In summary, he notes that not all trials are created equal. If the drugs are effective (and even Turner's re-analysis suggests that they are), then well designed studies will be more likely to show positive results than poorly designed studies. If well designed studies are also more likely to be published, then Turner's observation of higher publication rate for positive studies is a logically necessary result.
There is a bit of a bind here. When results depend on the interpretation of experts, they are open to criticism of bias. But on the other hand, a foolish consistency is the hobgoblin of small minds. The balance of the data seem to suggest that it was correct for the authors of inconclusive studies to interpret their findings positively - these drugs
are effective. Myself, I favor incorporating human judgment in the process and prefer to combat bias through transparency.
Kramer notes in a sidebar that
Turner's judgments, regarding spin in published data, are demanding; if researchers aggregated data, combining a highly positive study and an inconclusive one into a pooled analysis yielding overall moderate results, Turner counts the inconclusive study as having been misreported, even when the monograph indicates up front that data sets have been merged.
This approach has its strengths and weaknesses - and is fundamentally what Turner has himself done. What is gained by condemning it as universally inappropriate? Furthermore, there are good methodlogical reasons for preferring to publish positive findings over null findings. A scientific study is basically a line of reasoning. When the results all line up, we have some reason to think that all links in that chain of reasoning are correct. However, when the conclusion is not supported, the problem could be any single one of those links: maybe the outcome was poorly measured, maybe the entry diagnoses were inaccurate, maybe compliance to the regimen was low or maybe it was something else.
The bottom line, Kramer claims, is that the FDA procedures still seem to have made the correct decision about these drugs.
Nor is it at all clear that the standards the FDA used for drug approval yielded harmful results for public health purposes. One requirement was that a drug must demonstrate efficacy in two trials; inconclusive findings in other trials might then be set aside. This standard may have been reasonable. Given how much "noise" enters into even careful research—problems in diagnosis, problems in outcome measurements—it is hard to identify effective medications. Only about one in 10 drugs that enter late-stage trials comes to market; excluding effective drugs may be a price we pay for setting the bar as high as we do. The Turner reanalysis supports this "two is enough" policy. When you include the negative data, each of the 12 antidepressants under study still demonstrates efficacy. That's not to say that full study results shouldn't be made public—only that there's no evidence that the old procedures led the FDA astray.
