The scientific literature paints an overly rosy picture of the efficacy of psychological treatments for depression.
That is the conclusion of a study published Sept. 30 in the journal PLOS ONE. It is the follow-up to a study published in 2008 that created a considerable stir when it found a comparable publication bias in scientific articles reporting the efficacy of antidepressant drugs.
“[rquote]This doesn’t mean that psychotherapy doesn’t work. Psychotherapy does work. It just doesn’t work as well as you would think from reading the scientific literature[/rquote],” said Steven Hollon, Gertrude Conaway Vanderbilt Professor of Psychology, who co-authored the study with colleagues from Oregon Health & Science University, VU University Amsterdam and the University of Groningen.
The basic problem arises because clinical studies of the treatments for depression with more positive outcomes are more likely to be published than studies with less favorable results. “It’s like flipping a bunch of coins and only keeping the ones that come up heads,” Hollon said.
The research team identified all the U.S. National Institutes of Health grants awarded to fund clinical trials of psychological treatments for depression from 1972 to 2008. They found that nearly a quarter of these grants (13 of 55) had not published trial results.
They contacted the researchers who had conducted the 13 unpublished studies and requested the results from their studies. Using the unpublished data together with the published data, they conducted a series of meta-analyses from which they concluded that psychotherapy works, but that its effectiveness was inflated by publication bias.
“[lquote]This study shows that publication bias occurs in psychotherapy, mirroring what we’ve seen previously with antidepressants and other drugs[/lquote],” said co-author Erick Turner, associate professor of psychiatry and pharmacology at OHSU School of Medicine, who directed the 2008 study of antidepressants.
(A question that was raised, but not answered, by that study was whether it was reasonable to recommend psychotherapy over drug treatment without examining whether publication bias might be occurring with psychotherapy, too.)
“Journal articles are vetted through the process of peer review, but this process has loopholes, allowing treatment benefits to be overstated and potential harms to be understated,” Turner said. “The consumers of this skewed information are health care providers and, ultimately, their patients.”
The authors suggest that both the funding agencies and the journals should archive the original proposals and raw data from the trials – both published and unpublished – so that this form of reporting bias can be detected and corrected in the future.
Ellen Driessen, assistant professor of clinical psychology at VU University Amsterdam was the lead author on the article (though not available for comment at the time of publication). In addition Pim Cuijpers, professor of clinical psychology at VU University Amsterdam, and Claudi L.H. Bockting, professor of clinical and health psychology at the University of Groningen, also co-authored the study.
This project was conducted without grant support.