Scoop Protection Helps Ensure Reliability of Science

Researchers at the University of Technology Eindhoven (TU/e) and the Arizona State University have developed a new model to better understand the rules and procedures meant to promote innovative research and see if current reforms to improve the system actually make sense. Their study was published today in Nature Human Behaviour.

"Over the past decade, there have been growing concerns that something is ‘rotten’ in the state of science," says Leonid Tiokhi, lead author of the paper. "Scientists are realizing that many rules and procedures are dictated by norms and historical precedent, rather than principled reasons that serve efficiency and reliability. Worse, there is growing evidence that some norms have adverse side-effects that can hurt both science and scientists."

"Many scientists have sleepless nights worrying about being 'scooped'—fearing that their work won't be considered ‘novel’ enough for the highest-impact scientific journals," says Tiokhin. The priority rule has been around for centuries. "This is partly why some academic publishers, such as PLOS and eLife, decided to offer 'scoop protection', allowing researchers to publish findings identical to those already published within a certain timeframe. The problem is that we don't yet have a good idea about whether these reforms make sense."

To figure out the effects of rewarding priority (and whether recent reforms offer any solution for its potential drawbacks) Tiokhin and his colleagues developed an evolutionary agent-based model. This computer model simulates how a group of scientists investigate or abandon research questions, depending on factors such as the type of results (positive or negative) and the novelty of the research (the cost of being 'scooped').

The researchers found that a culture that rewards priority—where there is a big cost to being scooped—can have harmful effects. Among other things, it motivates scientists to conduct 'quick and dirty' studies, so that they can be first to publish. This reduces the quality of their work and harms the reliability of science as a whole.

The model also suggests that some form of scoop protection, as introduced by PLOS and eLife, works. "It reduces the temptation to rush the research and gives researchers more time to collect additional data," says Tiokhin. "However, we should keep in mind that scoop protection is no panacea."

"So while scoop-protection reforms are helpful, they are not sufficient to incentivize high-quality research or a reliable published literature," according to Tiokhin. "Our results suggest that we should also consider having some form of start-up costs, such as asking scientists to pre-register their studies or get their research plans criticized before they begin collecting data."