Newswise — Confirming suspicions that studies with low-key results often get neglected, a review finds that research is more likely to end up in print if it has a certain 'wow' factor.
Based on the findings of five studies, a new Cochrane systematic review estimates that trials are 1.78 times more likely to be published if they are perceived as important, reveal a positive effect or offer scientifically significant findings.
Kay Dickersin, a co-author of the review, said this is no minor matter. "If positive results are published more often than negative, what we think we know isn't really what we know. We might think a drug works, when it really doesn't work, because the negative results haven't been published," said Dickersin, director of the U.S. Cochrane Center at John Hopkins University's Bloomberg School of Public Health.
Scientists have spent hundreds of years worrying about "publication bias," Dickersin said, but researchers only began tackling the issue in depth during the 1990s. The new review examined five studies from the 1990s, including some by Dickersin herself.
The review appears in the latest issue of The Cochrane Library, a publication of The Cochrane Collaboration, an international organization that evaluates medical research. Systematic reviews draw evidence-based conclusions about medical practice after considering both the content and quality of existing medical trials on a topic.
All of the studies found that publication was more likely for trials with positive findings. The studies suggested that 41 percent of negative trials — those that show a drug or treatment has bad effects or none at all — make their way to print. By contrast, the researchers estimated that journal editors publish about 73 percent of positive studies.
Two studies looked at how long study publication took. One found that positive studies showed up in journals an average of 4.3 years after initial enrollment of patients, compared with 6.5 years for studies with negative findings. The other study, which took a slightly different approach, reported an even larger gap.
The results are not surprising to those familiar with the medical research world, but "we now have an estimate of how big the problem is, combining all the studies that examined publication bias in the same way," Dickersin said. "It's empirical data and no longer a hunch that we have had for several hundred years."
It is unclear from the data, however, whether investigators fail to submit negative studies or journal editors prefer not to accept them.
What should the research world do? Dickersin said it is vital to create registries that include all studies that are in the works. "Unfortunately, there are very powerful lobbies, and there's inertia that has prevented registration of all human trials in all countries," she said.
Still, trial registration does not solve the whole problem, she said, because publication does not always follow registration for study outcomes or results. That means the world might know there was a study, but never find out what the researchers discovered.
Dr. Trish Groves, deputy editor and senior research editor of BMJ, formerly known as the British Medical Journal, said the findings of the review sound valid. "However, the usefulness of the review is limited by its inability to add much to what we already know or to shed light on current editorial decision-making and practice," she said.
Editors at BMJ try to avoid publication bias by focusing on the question asked by a study under consideration and the quality of the methods used to find an answer, Groves said. "Whether the answer was positive or negative takes relatively low priority."
Groves added that two of BMJ's top 10 research papers in 2008 had negative results. (The journal chose the list using a variety of criteria, including citations, downloads and letters about the studies.)
As for the future, review author Dickersin said researchers should do more to examine how factors like language and funding sources affect the publication of a study.
Researchers should also consider ethical issues regarding patients who take part in studies, she said. "If patients had participated thinking they'd contributed to knowledge and then that study is never published, we've broken the covenant with the patients. That's the ethical side effect of the failure to publish."
The Cochrane Collaboration is an international nonprofit, independent organization that produces and disseminates systematic reviews of health care interventions and promotes the search for evidence in the form of clinical trials and other studies of interventions. Visit http://www.cochrane.org for more information.
Hopewell S, et al. Publication bias in clinical trials due to statistical signiï¬cance or direction of trial results. Cochrane Database of Systematic Reviews 2009, Issue 1.
RSS