Analysis of Clinical Study Reports raises questions about heart trial results

A group of authors, who are part of an international effort called RIAT, (Restoring Invisible and Abandoned Trials) have published a re-analysis of a major trial of a cholesterol-lowering drug.

Their findings have been called both important and not real science. You are interested now, I hope.

Meet a pivotal heart disease study

The story begins in 2017, when the New England Journal of Medicine published the FOURIER trial comparing a drug called evolocumab vs placebo in more than 27,000 patients who already had established heart disease and a persistently elevated LDL-C despite taking statins.

Evolocumab is an inhibitor of proprotein convertase subtilisin/kexin type 9. Suffice to say these injections markedly reduce levels of cholesterol—which should be important in patients who already have heart disease.

Here is a picture from the FOURIER trial. You can see that the drug markedly reduced LDL-C over placebo

In cardiology, it’s not enough for a drug just to change a level of something in the blood. You need it to improve outcomes.

In FOURIER, the primary outcome was a composite of 5 things: the first event of cardiovascular death, MI, stroke, unstable angina or the need for coronary revascularization (stent or bypass).

FOURIER reported that evolocumab reduced the incidence of the primary composite endpoint by 15%. We write that as a hazard ratio with 95% confidence intervals (HR 0.85 (95% CI, 0.79 to 0.92)).

Puzzling trial results

On the surface, this looked like a win for the drug. FDA approved it for use.

There were, however, some curiosities regarding these results.

Death due to cardiovascular disease and overall death were slightly higher in the patients who took evolocumab. Not (statistically) significantly higher, but I found it curious that a drug that demolished cholesterol levels, reduced the chance of a heart attack, stroke or the need for stents, did not result in less death due to heart disease or less overall death.

A deeper dive via Clinical Study Reports

Now to the new report from the RIAT group.

These authors, one of whom I know personally, got hold of the Clinical Study Reports—so called CSRs, which provide more details than that published in the NEJM. CSRs are technical documents prepared by the manufacturer and submitted to regulatory agencies as part of the approval.

They contain info about the trial’s protocol, amendments, inclusion and exclusion criteria, outcome definitions and measurement, efficacy and safety results and statistical analysis plan. The main FOURIER CSR is over 25 000 pages.

The first story is how these authors got hold of such documents.

They made numerous attempts to contact the authors and got no response. They asked FDA and FDA said it would take years. They eventually got access from Health Canada.

What did the researchers find?

Here were the main findings of the re-analysis:

• FOURIER had 870 deaths; 41% of the time the cause of death determined by the clinical events committee differed from that declared by the local investigator.

• The CSR info when compared with the 2017 publication in the NEJM, yielded 11 more deaths from MI in the evolocumab group vs 3 less death from MI in the placebo group.

• In the CSR, the number of deaths due to cardiac failure in the evolocumab group was almost double those in the placebo group (31 vs 16).

• The reported HR for cardiovascular mortality in the original trial analysis was 1.05 (95% CI 0.88 to 1.25). A 5% increase. In the re-analysis, it was now 20% higher in the evolocumab group. Technically, this was not statistically significant because the 95% confidence intervals ranged from 0.95 (a 5% decrease) to 1.51 (a 51%) increase.

The RIAT authors concluded that clinicians should be skeptical about prescribing evolocumab for patients with established atherosclerotic cardiovascular disease.

One study - two views

There are two views of this new study.

One is that there is nothing to see. A group of authors who had less information than the trialists looked at the CSR’s and came up with differences in causes of death. And, statistically speaking, the results of the trial were upheld.

Journalist Patrice Wendling has incredible coverage on theHeart.org | Medscape Cardiology website. Primary author of FOURIER, Marc Sabatine, said this of the new paper:

It's hard to call this science. I think it lacks all scientific rigor and is fundamentally flawed and, because their process was flawed, it has led them to erroneous conclusions."

Another view of this matter is that there is a lot more “flux” in the results of a major trial than I would have expected.

If the re-analysis, using CSRs, is correct, then cardiovascular death rates are even higher in the evolocumab group. The higher rate was weird in the main trial, and it’s weirder yet, in the re-analysis.

Finally, a colleague of mine from Egypt, Professor Ahmed Bendary, Tweeted that this might be the biggest story of 2023.

The paper he linked reported that mice without the molecule that evolocumab inhibits may be more susceptible to heart failure with normal heart function. It’s an animal study so let’s not over-react. But…

What does this tell us?

I take two main messages from this story:

Yes, it is possible that that the higher death rates from cardiac disease (in both the main trial and reanalysis) were just noise.

But perhaps it’s because drug effects are complex and cholesterol reduction may be only one effect of these drugs. The higher rates of cardiac failure in the reanalysis does comport with the animal study on PCSK9-deficient mice.

It is really hard to understand how this much cholesterol reduction, and MI and stroke reduction, and reduction in the need for stents or bypass, does not improve survival.

The other message is more wide-ranging—and more important:

Independent and transparent scrutiny of medical literature is crucial—especially when it leads to changes in practice.

Source data should be made available for independent review. Why would the FDA and authors refuse to share this data?

I’ve always held that great results will pass muster in independent analyses.

It shocks me that there is so little independent scrutiny of major medical evidence.

Note: This guest blog by John Mandrola, MD was originally published on Substack, where interested readers can subscribe to his Sensible Medicine newsletter. The original title was "Yet Another Story on Re-Analyzing a Major Trial". The author can be contacted on Twitter. TranspariMED added some sub-headings to the blog above. TranspariMED thanks John Mandrola for permission to reproduce his post here.

ICYMI: The European Medicines Agency recently announced that it will soon resume making Clinical Study Reports available to independent researchers.