New study shows importance of trial registries in reducing research waste

A new study indicates that reporting clinical trial results on registries may reduce research waste in medical research by around 40%.

Working from ethics approval documents for 326 clinical trials in multiple locations across four countries, the study team found that 13% of trials had never made their results public in any form.

However, relying on publication in an academic journal alone would have generated far more research waste, leaving 22% of trial results unreported. Thankfully, the team could find 40% of the results that were missing in the scientific literature on trial registries, pushing the research waste rate down to 13%.

Failing to pre-register clinical trials or make their results public is a clear violation of global medical research ethics under the Declaration of Helsinki and wastes public money, slows down medical innovation, and harms patients and public health.

Academia performs worse than industry

The study noted that only 16% of academic trials had made results available on a registry, compared to 84% of industry trials.

The WHO recommends that all trial results should be made public on registries, but many academics still have not adopted this best practice.

Consistent with previous studies, the researchers also found that complete non-reporting of results was far more widespread in academia (24%) than in industry (4%).

The study also assessed protocol quality – it found that better designed trials were more likely to make their results public – and captured reasons for trial discontinuations.

Is research waste worst in Germany?

Working from ethics approval documents for 326 clinical trials in multiple locations across four countries, the study team found that 19% of trials in Germany had never made their results public, compared to 16% in Switzerland, 14% in Canada, and 12% in the UK.

In terms of prospective trial registration, German trials (90%) performed better than Canadian (89%), Swiss (84%) and UK (81%) trials.

Note: These country-level breakdowns should be taken with a pinch of salt (see methodology discussion further below).

Surprising findings

The findings of the new study are surprising because multiple other studies have found far higher rates of non-reporting for clinical trials.

For example, a recent study of 1,658 German university trials listed on trial registries found a non-publication rate of 57% after two years. Another recent study of 4,657 paediatric trials run worldwide found that 48% had never made their results public.

Also, the new study found that across countries, only 6% of trials remained completely unregistered, a surprisingly low rate.

All 326 trials included in the study were approved by ethics committees in 2012. The cohort was almost evenly split between industry-sponsored and academic trials. Pilot, feasibility, and phase 1 studies were excluded.

Methodological maze

(If you are not a meta-research geek, you can skip the rest of this blog.)

All three studies cited above involved both literature and registry searches, and the paediatric study also included industry trials, so these factors cannot explain the large discrepancies in outcome reporting detected.

So which factors can?

The new study has considerable strengths.

Notably, the team worked from research ethics approval documents, a ‘gold standard’ methodology that is extremely time-consuming but captures all trials. In contrast, registry-based studies (such as the German university and paediatric trials studies cited above) inevitably miss trials that were never registered in the first place.

Another significant strength is that the team made efforts to identify and exclude trials that had never been started following ethics approval:

“In February 2022, we rechecked the trial status and availability of study results for all included RCTs. We designated the trial status as unclear when an RCT was labeled as ongoing in the registry but the status had not been updated in the previous 2 years, unless the planned completion date was after February 2022. We contacted RECs and surveyed principal investigators for clarification when trial status was unclear.”

(Registry data typically includes many ghost trials whose records are never updated after the initial registration, leaving their status unclear, so purely registry-based approaches can miscategorise essentially non-existent trials as completed but non-reported.)

Furthermore, the teams followed trials up a whole decade after their approval; any trials that remain unreported at that point have almost certainly become research waste. In contrast, some – but by far not all – of the German university trials left unreported two years after completion will probably make their results public at some point in the future.

The new study has a major limitation.

Its ethics approval based methodology involved convenience cluster sampling, and limited cohort sizes. For example, the team only included 37 trials from Germany, all of which were approved by the same ethics committee, and 35 trials from Canada. Swiss trials (n=165) accounted for half of the total sample, and UK trials (n=89) for the remaining quarter.

In sum, this study makes a very valuable contribution to the field, partly due to its methodological strengths, but above all because its surprising findings pose an interesting challenge to the meta-research community.

You can access the study here.

Full disclosure: While working as a Research Fellow for the QUEST Center for Responsible Research in Berlin, the author of this blog contributed to the German university trials study cited above, which found a far higher rate of non-reporting than the study discussed in this blog did.