CLAIMS of “fake news” within the UK Houses of Parliament are nothing new. This time, however, the charge has been laid not at the door of politicians, but of scientists. And it was scientists themselves making the claims. They came at a meeting I attended last week where the British Neuroscience Association (BNA) launched a fightback against bad science with its .
The problem isn’t just that some findings turn out to be wrong. It is, after all, the point of science to be constantly questioning, testing and refining hypotheses. The BNA campaign claims that the entire structural edifice of academia now encourages mistakes to be made.
Advertisement
This starts with well-meaning efforts by managers and funders to judge researchers’ productivity. That is done by gauging how many papers they write and the prestige of the journals that publish them, as quantified by their “impact factor” – basically, an average of how often the papers they publish are cited by other papers.
Researchers’ publication records increasingly govern every aspect of their career success, including pay rises, future jobs and funding for new projects. In this “publish or perish” culture, it is in their interests to produce a blizzard of papers that are groundbreaking and flashy, so as to get published in high-impact journals. With an eye to maintaining their impact factor, journals are incentivised to publish such papers, rather than ones that, for example, describe attempts to replicate others’ work.
The resulting system is the antithesis of how good science should be done, namely by tackling questions in a thoughtful and systematic way and by testing and retesting any unexpected result in different labs and circumstances. Prime evidence of how bad things have become is the replication crisis in psychology, where doubts have been raised over classic findings such as priming, the idea that behaviour can be changed by subtle, unconscious cues. In psychiatry, a review published this year called into question two decades of work on a . “It wasn’t just that people said it mattered and it didn’t, it’s that we built whole castles in the air on it mattering,” said psychologist Dorothy Bishop of the University of Oxford at the BNA event.
The campaign is a laudable attempt to change this underlying culture. As Bishop said, “We have to do things in a way that ensures discoveries are robust.” The aim is to do that by lobbying universities and funders, as well as by training scientists in best practice, such as data sharing and registering studies before publication, meaning mistakes are more likely to be noticed.
Happily, it isn’t a lone initiative. The recently launched . Growing numbers of bodies are signing the , made at a cell biology conference in 2012, to say they won’t use journal impact factors in decisions on funding and job appointments.
If the movement succeeds, that would mean fewer interesting stories for journalists like me to write about, but the ones we do cover would be more likely to be true. “If everyone’s trying to do groundbreaking research, you just end up with a lot of holes in the ground,” said Bishop. “You don’t get anything built.”
