Evidence distortion in medicine explained in one single chart

Back in 2008, Erick Turner and colleagues raised eyebrows when they published a study of 74 clinical trials of antidepressant drugs and found that the scientific literature gave only a limited – and highly distorted – picture of what really happened during those trials.

The reason? While the results of nearly all ‘positive’ trials that showed a drug to be effective were published, a significant share of ‘negative’ trial results were never made public. In addition, a handful of ‘negative’ trials were misleadingly misrepresented as ‘positive’ in medical journals.

The 2008 study became an instant classic of its genre and has since been cited over 1,400 times.

BACK TO THE FUTURE

Now Erick Turner has returned to his old hunting grounds with a fresh team to check whether things have improved over the past decade.

The image below shows what he found.

Out of 30 antidepressant trials the team looked at, 15 were ‘positive’ and 15 were ‘negative’ (top row).

The scientific literature – the place where doctors look for guidance on how well drugs work – presented a rather different picture: 17 ‘positive’ trials versus only 7 ‘negative’ trials (bottom row).

The difference is due to six ‘negative’ trial results never being published, and a further two ‘negative’ trials being misrepresented as ‘positive’ in print.

The team’s verdict: The medical evidence base is still being systematically distorted, albeit to a lesser degree than a decade ago.

WHERE IS THE FDA?

This problem is not limited to antidepressants. It has been exhaustively documented in countless studies cutting across many areas of medicine.

Congress tried to create a more reliable evidence base in 2007, when it passed a law requiring many (but by far not all) clinical trials to disclose their results in a public registry.

However, over 3,000 trial results are still missing from the registry today, and the FDA has yet to impose a single fine. So far, all it has done is send out a few warning letters – at a snail’s pace.

As a result, companies and institutions across America continue to break the law with impunity, leaving doctors flying blind and patients unable to make informed treatment choices.

This is a problem that could easily be solved.

The solution is simple: The FDA must start enforcing the law.

Note: The blog above is an abridged version of an article first published on the Mad in America website.