Cochrane review: 47% of all clinical trial results are not made public

Nearly half of clinical trial results are not made public, an exhaustive review published by the medical evidence group Cochrane has found.

The new Cochrane review draws together data from 204 existing studies assessing the (non-)publication of results for 165,135 clinical trials in scientific journals, making it the most comprehensive effort of this kind to date.

Failing to make the results of clinical trials public leaves gaps in the medical evidence base that can harm patients, undermine public health and inflate the costs of healthcare.

Key findings:

• The results of 47% of trials were not made public

• Industry was worse at publishing trial results than academia or public bodies

• Median time to publication was over 2 years after trial completion

Additional findings on publication bias:

• Trials with ‘positive’ results were more likely to be published

• Trials with ‘positive’ results were typically published more rapidly

According to the authors:

“When we limited our analysis to well‐conducted studies, the likelihood of publication slightly increased to 57%... This was potentially due to more robust methods of searching for publications.”

Image: Screengrab from the Cochrane review

Many major research funders in the United States and worldwide have in recent years begun checking up on whether their grantees publish trial results within the one-year time frame.

The authors note that they have been unable to detect improvements in reporting over time. However, they only assessed studies published up to August 2023. As the studies themselves typically look several years back in time, their methodology is unable to capture possible recent improvements (or declines) in reporting.

What is the way forward?

TranspariMED reached out to the lead author, Marian Showell, to ask about the broader implications of her findings.

Q: Does this only measure journal reporting, i.e. did you disaggregate findings reported within individual studies when including studies that measured both journal and registry reporting?

A: If a research report presented only the results posted on a registry it was excluded (18 studies). If it wasn't possible to disaggregate the data, we used the combined posted and published data (5 studies). If the study reported separate data for published and results posted, we extracted and used only the publication in journal data.

Q: You mention that, impressionistically, reporting rates had not improved over time. Why did you not analyse that rigorously?

A: We sorted the rate of publication forest plot by year and assessed it visually as we found this to be an interesting finding, however this was not the main aim of our review, so we didn’t investigate this further.

Q: Did you also get a sense of regional variations, e.g. Europe vs United States vs China vs India?

A: No, sorry.

Q: What do you estimate true non-reporting rates are, considering that some trials are never registered and other trials never start?

A: I wouldn't like to make a guess.

Q: Where do you think further research is not required, and where do you think the community should focus future efforts?

A: Further research into publication rates and time to publication in the style of this systematic review is not necessary and could constitute research waste.

However, examining more recent research reports on publication which only include recent cohorts may more clearly reveal the effects of initiatives like AllTrials.

Future research efforts in this area could focus on the impact of new incentives or models aimed at improving the dissemination of results and reducing publication bias. For example, the appointment of research managers responsible for trial management or the UK ethics model, where all approved trials are passed to the Health Research Authority (HRA), which registers the trials and checks for the reporting of results one year after trial completion.

[Note by TranspariMED: The UK Health Research Authority now captures all trials registered in the UK, but has still not delivered on its long-standing promise to also check up on their reporting status.]

Further research on the results deposited onto a registry platform would also be useful to determine whether this…

1. Improves dissemination rates, decreases publication bias and improves the time to dissemination when compared to journal publication?

2. Are results posted as rigorous as published [in a journal] data or vice versa?

3. Do results posted on a registry platform negate the huge problem of fraudulent publications or not?

Further research into the problem of non-dissemination of results and publication bias should focus on automated linking of all clinical trial stakeholders in the dissemination pipeline i.e. funders ethics, regulatory bodies, institutions, and researchers.

How do we achieve this i.e. through regulation/legislation? How to make this an international initiative?  What sort of designated group/committee should ultimately be at an international level?

Q: Does the lack of improvement over time indicate that whatever people have been trying to date to improve reporting has not worked? What are the lessons learnt in this field?

A: Making publication or dissemination of results the sole responsibility of the author hasn't worked which is why the UK model of the ethics committees taking responsibility for registration and results is very interesting.

The work towards prospective trial registration has been monumental in providing transparency and ideally minimising publication bias by selective reporting. This was and remains an important step along the way towards 100% publication, but we now need to go further.   

All results should be made public within one year

Global frameworks are converging on a results publication timeline of one year after trial completion.

The Cochrane review found that median time to publication was over 2 years after trial completion - which slows down scientific progress and the introduction of better treatments. (However, the review focused exclusively on publication in scientific journals, and did not take into account results posted onto registries.)

According to the 2024 update of the Declaration of Helsinki, which sets out global ethics rules for medical research, all trial results have to be made public in a “timely” manner. The World Health Organisation requires publication of all trial results within one year of trial completion.

Laws in multiple jurisdictions including the EU and US also require results publication within one year, but such laws focus narrowly on uploading results onto registries (which the Cochrane review did not assess) as opposed to publication in scientific journals. Also, existing laws only apply to some types of trials, rather than covering all interventional trials.

The Cochrane review “Time to publication for results of clinical trials” has been published open access and is available online. Readers interested in improving trial reporting at their institution or within their country can find useful resources on the TranspariMED website.