SDC-1802: Why the design of trials is so critical23 Dec 2021 19:14
From the latest issue of New Scientist:
"More than half of cancer biology lab findings cannot be replicated.
A long investigation into the reliability of preclinical cancer biology research has found that fewer than half of the results published in 23 highly cited papers could be successfully reproduced.
Tim Errington at the Center for Open Science in Virginia, which conducted the investigation, says the original plan was to reproduce 193 experiments from 53 papers. However, this was reduced to 50 experiments from 23 papers.
“Just trying to understand what was done and reported in the papers in order to do it again was really hard. We couldn’t get access to the information,” he says.
The 50 experiments included 112 potentially replicable binary “success or failure” outcomes. But Errington and his colleagues could replicate the results of only 51 of these – or 46 per cent (eLife, doi.org/g8tc, doi.org/gnp846).
The experiments were all in-vitro or animal-based preclinical cancer biology studies, and didn’t include genomic or proteomic experiments. They were from papers published between 2010 and 2012 and were selected because they were all “high-impact” studies that had been read and heavily cited by researchers.
The findings of the eight-year investigation align with earlier reports from pharmaceutical companies Bayer and Amgen. C. Glenn Begley was a cancer biologist at Amgen and an author of its report, published in 2012. “We looked back at the papers that we had relied upon at Amgen and found that we could only reproduce 11 per cent of the studies,” says Begley.
The Amgen report was applauded by some in the research community for shining a light on an important problem. But Begley says it was also criticised for a lack of openness about which studies it tried and failed to replicate.
This criticism can’t be levelled at the new investigation. Errington and his colleagues have published all the data about the studies they included. They also invited peer review of their methods before the study ended.
The investigation focused on preclinical studies, but the problems it uncovered might help explain issues with later-stage studies in people too. A previous survey showed that less than 30 per cent of phase II and less than 50 per cent of phase III cancer drug trials succeed.
There are signs of change on the horizon. The US National Institutes of Health is instituting a new policy in early 2023 that will make data sharing the default for the many projects it funds. Several journals have also changed their publishing systems in recent years to encourage open science and data sharing."
Whilst the above focuses on papers published 10yrs ago, it illustrates why it is so important for trials to be carried out under the most robust and reliable methodologies. If SAR should require more time to ensure future trials are up to scratch then that's fine with me. Exploring the frontiers of medicine shouldn't be rushed.
Sign In
Follow the stocks
