Many cancer studies stealthily move goalposts, new study finds

Nearly a fifth of cancer trials change their primary endpoints after study launch, and in most cases such changes are not disclosed in scientific publications, a large study of 755 cancer trials has found.

It appears that primary endpoints were sometimes changed to make the treatments under investigation appear more effective than they really were: 61% of trials with changed outcomes had ‘positive’ results, versus only 51% of trials that used the originally specified success criteria.

There can be legitimate reasons for changing a trial’s endpoints after it has started. However, such moving of the goalposts should always be reported in scientific publications. Over 70% of cancer trials that changed their endpoints failed to disclose that the goalposts of the study had been moved.

The studies raises concerns that doctors do not have an accurate picture of the effectiveness of some of the cancer treatments that they prescribe.

Multiple red flags

Another worrying finding was that 63 trials had changed their endpoint on ClinicalTrials.gov after the trial had been completed.

It is unclear which of those cases are due to shoddy registry management practices, and which are due to outright manipulation.

In addition, nearly two thirds of trials had not made their protocols public. (Study protocols provide detailed insight into how a trial is designed.) Therefore, the team may not have been able to detect all cases in which outcomes were changed.

How can this be fixed?

The authors recommend that scientific journals improve their policies and processes.

However, there are thousands of journals worldwide, each setting (and often not following) their own rules.

Multiple previous studies have found that scientific journals cannot be trusted to provide an honest and accurate record of the benefits and harms of new treatments tested in clinical trials. Most recently, 41% of a large cohort of German university trials were found to have primary outcome discrepancies between the latest registry entry and the publication.

TranspariMED believes that a more effective way to curb evidence distortion in medicine is to require researchers to archive study protocols and report results on clinical trial registries.

Relevant World Health Organization best practices also focus on registries, as do laws in Europe and the United States.

Where are the data?

This study is methodologically very strong.

The authors included all completed phase 3, cancer-specific, interventional RCTs registered on ClinicalTrials.gov up to February 2020 for which results had been reported in journals. They used registry records, journal articles and (where available) study protocols to detect outcome changes.

A key weakness is that the authors did not make their dataset public alongside their scientific paper.

The dataset would be useful to examine which drugs were involved, whether trials run by industry or by non-commercial players were more likely to publish protocols or change their endpoints, to identify which journals did and did not ensure that outcome changes were disclosed, to dig deeper into those trials that appeared to have changed their endpoints after the study had been completed, and to answer numerous other questions that could help to ensure that cancer patients are receiving the best possible treatment.

The lead author of the study informed TranspariMED that “we are currently compiling data for sharing purposes as requested by you and others.”

While TranspariMED applauds the team’s commitment to transparency, the question remains why JAMA Network Open does not routinely publish raw data together with such meta-research papers in the first place.