UK public research funders show mixed performance on clinical trial transparency

One year after three UK public research funders agreed to adopt the WHO’s strong standards on clinical trial transparency, it appears that none of them has fully implemented their commitments.

Responses to Freedom of Information requests filed by TranspariMED indicate that two funders, MRC and NIHR, have taken significant steps towards greater transparency, but still fall short of fully delivering on the promises they made. The third funder, DFID, has taken no action to meet its commitments.

Key commitments made one year ago

At the time, the May 2017 WHO Joint Statement was strongly welcomed by transparency advocates, with AllTrials co-founder Dr Ben Goldacre hailing it as “truly fantastic news”.

The signatories to the statement promised that:

“Within 12 months of becoming a signatory of this statement, we each pledge to develop and implement a policy with mandated timeframes for prospective registration and public disclosure of the results of clinical trials that we fund, co-fund, sponsor or support.”

Concrete commitments included:

Trial registration: “Before any clinical trial is initiated (at any Phase) its details must be registered in a publicly available, free to access, searchable clinical trial registry”

Summary results posting: “We will work towards a timeframe of 12 months from primary study completion as the global norm for summary results disclosure… this joint statement focuses on use of registries – such as clinicaltrials.gov and EU-CTR - to meet this results disclosure expectation… Clinical trial registry records should be updated as necessary”

Transparent audits of compliance: “We each agree to monitor registration and endorse the development of systems to monitor results reporting on an ongoing basis… We agree that transparency is important and therefore the outputs from the monitoring process will be publicly available.”

Which UK funders have delivered on their promises?

Department for International Development (DFID)

DFID has not taken any steps to meet its commitments. In its Freedom of Information response, the foreign aid arm of the UK government explained that:

“The Department for International Development (DFID) has not conducted audits of clinical trial registration/ reporting. DFID supports clinical trials in partnership with other funders. Compliance with clinical trial registration is assessed during the evaluation process, before any DFID funding is awarded and subsequently during our annual review process. The vast majority of DFID implementing partners are themselves signatories to the WHO statement.”

DFID’s response is extremely disappointing. It is unclear how DFID assesses compliance with trial registration requirements. Typically, trials are only registered after funding has been secured, and the results of DFID’s monitoring process are not publicly available. Also, DFID does not monitor reporting performance, instead passing the buck for doing so to its grantees by blindly trusting that its ‘implementing partners’ - i.e. grantees - themselves have put adequate monitoring systems into place. Ongoing research by the EBM Data Lab shows that this trust is misplaced (see further below).

In sum, if DFID considers it unnecessary to adopt policies on trial reporting and monitor compliance with them, why did it sign up to the WHO Joint Statement in the first place?

Medical Research Council (MRC)

The MRC has long been a frontrunner in clinical trial transparency. A recent blog by AllTrials justifiably praises the public body for being one of only two top trial funders worldwide who “hit the gold standard” in clinical trials transparency.

In its Freedom of Information response, the MRC flagged two excellent reviews of trial registration and results reporting performance by its grantees conducted since it signed the statement. However, against this positive backdrop, it is disappointing that MRC’s reviews seem to focus on academic publication alone.

The scope of MRC reviews does not include grantees’ maintenance of trial registry entries or the posting of so-called ‘summary results’ onto registries, despite the fact that the WHO Joint Statement explicitly “focuses on use of registries”. (An ongoing separate audit currently being conducted by the EBM Data Lab has also flagged this shortcoming.) Note in this context that ISRCTN, the registry preferred by the MRC, is unusual among public trial registries in that it does not have a dedicated function for uploading stand-alone summary results.

Also, MRC’s publicly available reviews only include summary data of grantee performance, so it is not visible which individual grantees ensure that medical research conducted with taxpayers’ money meet global research integrity and reporting standards. This may reflect the unfortunate view still widespread in the medical research field, where such transparency is often disparaged as ‘naming and shaming’, rather than welcomed as a positive tool enabling researchers who do the right thing to receive public credit for their diligence.

In sum, the MRC has demonstrated a strong commitment to clinical trial transparency, but there still is some way to go.

National Institute for Health Research (NIHR)

The NIHR has also long been a frontrunner in clinical trial transparency in global comparison, with a recent study awarding it full marks for key trial registration and results reporting policies. Also, in its Freedom of Information response, NIHR highlighted several studies it has conducted on its grantees’ performance on trial registration and reporting. These date back to 2012, illustrating that NIHR has long been ahead of its peers in this respect.

However, past NIHR monitoring reports fail to make public the performance of individual grantees. In addition, NIHR has not released any further monitoring reports since signing the WHO Joint Statement, and in its response noted that it “does not collect data in the format requested”, and indicated that it has no plans to do so in the immediate future. Thus, detailed institution-wide data on grantee performance is not being made available at regular intervals.

NIHR explained that:

“The majority of NIHR audit activities are completed as part of routine management of projects and fellowships, by NIHR Coordinating Centre rather than via a formal NIHR wide audit. It is our expectation that [that] practice will continue.”

NIHR referred to a public document outlining its approach to “ensuring full publication of

NIHR funded studies”. The document notes that “The NIHR is unusual in being a health research funder that publishes transparent, full accounts of every piece of commissioned research in its own free, open-access, permanently archived peer-reviewed journal,” which in itself is excellent.

However, existing NIHR policy seems to focus exclusively on academic publication, and apparently neglects to cover trial registry maintenance and the posting of summary results on registries.

In sum, NIHR has demonstrated a strong commitment to clinical trial transparency. The ongoing detailed funder audit by EBM Data Lab will show in more detail how well NIHR measures up to the WHO Joint Statement pledges it made last year, especially with regard to trial registry maintenance and summary results posting on registries.

Conclusion

Britain’s two top public medical research funders MRC and NIHR perform well with regard to clinical trial transparency in global comparison, and deserve praise for doing so.

Their overall positive performance is especially encouraging when contrasted with that of DFID and some of the other WHO Joint Statement signatories – including France’s public funder Inserm, the Research Council of Norway, and the Gates Foundation – who appear to have taken little or no action over the past year to deliver on their transparency pledges.

Both MRC and NIHR should now work to address the remaining (and significant) gaps related to mandating and monitoring the regular updating of clinical trial registry entries, and the timely posting of summary results onto registries, and make transparent which of their grantees put public medical research funding to the best use.

In December last year, Transparency International, Cochrane, CRIT and TranspariMED jointly recommended that governments should:

“require all public research funding bodies within their jurisdiction to adopt and expand on the provisions of the recent WHO-brokered ‘Joint statement’ by research funders, and ensure that they are fully implemented. In future, to help ensure that public funding for medical research actually benefits the public, government funders should only give taxpayers’ money to institutions and individuals that verifiably comply with best practices in clinical research. Taking this simple first step would deliver significant transparency gains at minimal cost.”

The data gathered by TranspariMED and EBM Data Lab shows that periodic monitoring of funders’ policies can help political decision-makers to track public research funders’ delivery on their promises, identify remaining gaps, and (hopefully) increase momentum towards greater transparency.

TranspariMED will continue to monitor the implementation of the provisions of the WHO Joint Statement by its signatories in the UK and beyond. In addition, TranspariMED will work with advocacy groups in other countries to encourage their governments to also direct public funders to sign up to the WHO Joint Statement and fully implement its provisions.

When the EBM Data Lab team have completed their ongoing, far more detailed review of all WHO Joint statement signatories, TranspariMED will summarize those international findings.

Keep tuned to this blog and follow TranspariMED on Twitter.