The quest: a better understanding of nature. Photograph: Sebastian Kaulitzki/Alamy
In an ideal world, scientific discoveries would be independent of what scientists wanted to discover. A good researcher would begin with an idea, devise a method to test the idea, run the study as planned, and then decide based on the evidence whether the idea had been supported. Following this approach would lead us step-by-step toward a better understanding of nature.
Unfortunately, the life sciences are becoming increasingly estranged from this way of thinking. Early in their training, students learn that the quest for truth needs to be balanced against the more immediate pressure to "publish or perish". For a junior scientist to compete at securing a permanent academic position, her top priority must be to publish in journals with the greatest prestige and impact. If she survives to become a senior scientist, she's likely to then pass this lesson on to her own PhD students.
This publishing culture is toxic to science. Recent studies have shown how intense career pressures encourage life scientists to engage in a range of questionable practices to generate publications – behaviours such as cherry-picking data or analyses that allow clear narratives to be presented, reinventing the aims of a study after it has finished to "predict" unexpected findings, and failing to ensure adequate statistical power. These are not the actions of a small minority; they are common, and result from the environment and incentive structures that most scientists work within.
At the same time, journals incentivise bad practice by favouring the publication of results that are considered to be positive, novel, neat and eye-catching. In many life sciences, negative results, complicated results, or attempts to replicate previous studies never make it into the scientific record. Instead they occupy a vast unpublished file drawer.
The scientific community is well aware of these problems – in fact, we've known about them for decades. The problem is that any one scientist opting to work beyond the system immediately disadvantages herself relative to her peers. The only solution is structural reform, and to this end some of us have recently taken steps to drive change.
Since May this year, the journal Cortex, a peer-reviewed outlet for science on the nervous system and behaviour, has offered authors the opportunity to publish a type of article called a registered report. Unlike traditional scientific publishing, in which manuscripts are peer reviewed only after studies have been completed, registered reports are reviewed before scientists collect data. If the scientific question and methods are deemed sound, the authors are then offered "in-principle acceptance" of their article, which virtually guarantees publication regardless of how the results turn out.
The journals Attention, Perception & Psychophysics and Perspectives on Psychological Science have launched similar projects. Both initiatives borrow from the now-established requirement that clinical trials pre-register their study protocols. But these new initiatives go even further by offering publication of the eventual results in the same journal, regardless of what is found.
By basing editorial decisions on the question and method, rather than the results, pre-registration overcomes the publication bias that blocks negative findings from the literature. And by conducting peer review both before and after a study is completed, questionable practices to increase "publishability" are greatly reduced. The aim here isn't to punish the academic community for playing the game that we created; rather, we seek to change the rules of the game itself.
Critics have argued that pre-registration is overzealous and will hinder exploration, meaning serendipitous findings would remain hidden to us. We agree that exploration is vital, but while this concern is understandable, it is also easy to guard against. For instance, the registered reports initiative allows authors to report on any aspect of their data – even when such analyses are not registered at the outset. However, these outcomes are clearly labelled as exploratory to make them distinct from the pre-planned analyses. Registered reports also require authors to publicly release their raw data so that other scientists can explore the results in unanticipated ways, now and in the future.
Our publishing culture is conservative and slow to evolve. Following the Cortex initiative, some of us have witnessed quiet resistance to pre-registration from other journals. These outlets fear that agreeing to publish papers before seeing the data could lock them into publishing negative results or other findings conventionally regarded as "boring". This is despite the fact that clear-cut negative outcomes can be tremendously informative, telling us which potential interventions don't work or which suspected phenomena don't actually exist.
The deeper concern of journals is that pre-registration threatens existing "prestige" hierarchies and could reduce a journal's impact factor – a metric that is arguably meaningless as an indicator of scientific quality and, in fact, predicts the rate of article retractions due to fraud.
Nobody can expect scientists to sacrifice their livelihoods or those of their proteges for the good of the cause. So, as a group of scientists with positions on more than 100 journal editorial boards, we are calling for all empirical journals in the life sciences – including those journals that we serve – to offer pre-registered articles at the earliest opportunity. The guidelines for the initiatives at Cortex and Perspectives are straightforward, and while specific kinds of studies will require specific solutions, the general principle is widely applicable. For pre-registration to benefit science it must be embraced by a broad spectrum of journals.
Study pre-registration doesn't fit all forms of science, and it isn't a cure-all for scientific publishing. But it is a crucial part of urgent wider reform. Our publishing culture is no longer fit for purpose and the time has come to offer scientists a genuine alternative to "publish or perish". If the life sciences are to preserve a legacy of truth, journals must welcome pre-registration with open arms.
Jump to comments
Prof Paul Aveyard, University of Oxford, UK (on editorial board of Addiction)
Prof Mark Bellgrove, Monash University, Australia (editorial board: Journal of Attention Disorders)
Dr Marco Bertamini, University of Liverpool, UK (editorial board: Acta Psychologica, Arts & Perception)
Dr Sven Bestmann, University College London, UK (on editorial board of Frontiers in Human Neuroscience)
Prof Dorothy Bishop, University of Oxford, UK (editorial boards: Autism, Laterality and former editor of Journal of Child Psychology and Psychiatry)
Prof Bjoern Brembs, University of Regensburg, Germany (editorial boards: of Frontiers in Decision Neuroscience, PLOS ONE)
Prof John Britton, University of Nottingham, UK (former editor-in-chief of Thorax)
Prof Anne Castles, Macquarie University, Australia (editorial boards: Cognitive Neuropsychology, Cortex, Journal of Experimental Child Psychology, and Scientific Studies of Reading)
Dr Chris Chambers, Cardiff University, UK (editorial boards: Cortex and PLOS ONE)
Prof Axel Cleeremans, Universite Libre de Bruxelles, Belgium (editorial board: Consciousness and Cognition, field editor-in-chief of Frontiers in Psychology and former editor of Psychologica Belgica)
Dr Molly Crockett, University College London, UK (on editorial board of Social Cognitive and Affective Neuroscience)
Dr Sean David, Stanford University, US (editorial board: Nicotine & Tobacco Research)
Prof Bill Deakin, University of Manchester, UK (editorial boards: Schizophrenia Research, Journal of Psychopharmacology
Prof Sergio Della Sala, University of Edinburgh, UK (editor-in-chief, Cortex)
Prof Zoltan Dienes, University of Sussex, UK (editorial boards: Cortex, Frontiers in Consciousness Researchm, Psychology of Consciousness: Theory, Research, and Practice)
Dr Simon Dymond, Swansea University, UK (editorial boards: Analysis of Gambling Behavior, Journal of Contextual Behavioral Science, Journal of the Experimental Analysis of Behavior, The Behavior Analyst, European Journal of Behavior Analysis)
Prof Klaus Ebmeier, University of Oxford, UK (editorial boards: Biological Psychiatry, Journal of Psychiatry and Neuroscience, International Journal of Neuropsychopharmacology, American Journal of Neurodegenerative Disease, Depression Research and Treatment)
Prof Gary Egan, Monash University, Australia (editorial boards: Human Brain Mapping, Journal of Neuroinformatics, International Journal of Imaging Systems and Technology, Frontiers in Neuroinformatics)
Dr Pete Etchells, Bath Spa University, UK (editorial board: BMC Psychology)
Prof Matt Field, University of Liverpool, UK (editorial boards: Addiction, Drug and Alcohol Dependence, Psychopharmacology)
Prof Gabriele Fischer, Medical University of Vienna, Austria (editorial board: Addiction)
Prof Jonathan Flint, University of Oxford, UK (editorial board: PLOS Genetics)
Prof Ingmar Franken, Erasmus University Rotterdam, Netherlands (editorial boards: Drug and Alcohol Dependence, Frontiers in Psychopathology, Journal of Addiction)
Dr Joy Geng, University of California Davis, US (editorial board: PLOS ONE, Frontiers in Human Neuroscience)
Prof Morton Ann Gernsbacher, University of Wisconsin-Madison, US (editorial board: Cognitive Psychology, Discourse Processes, former chief editor, Memory and Cognition)
Prof Bradley Gibson, Notre Dame University, US (editorial boards: Attention, Perception & Psychophysics, Journal of Experimental Psychology: General, Journal of Experimental Psychology: Human Perception and Performance, Visual Cognition)
Dr Sam Gilbert, University College London, UK (editorial board: PLOS ONE)
Dr Stephen Gilman, Harvard University, USA (editorial board: Nicotine & Tobacco Research)
Prof Justin Harris, University of Sydney, Australia (editorial boards: PLOS ONE, Journal of Experimental Psychology: Animal Behavior Processes)
Prof Matt Hickman, University of Bristol, UK (senior editor and deputy regional editor, Addiction)
Dr Lee Hogarth, University of New South Wales, Australia (editorial boards: Frontiers in Cognition, Quarterly Journal of Experimental Psychology)
Dr Alex Holcombe, University of Sydney, Australia (editorial board: Perspectives on Psychological Science)
Dr Hans IJzerman, Tilburg University, The Netherlands (special issue editor, Frontiers in Cognition)
Prof John Ioannidis, Stanford University, US (editorial boards: Lancet, PLOS Medicine, Journal of the National Cancer Institute, Science Translational Medicine, Molecular and Cellular Proteomics, International Journal of Epidemiology, Cancer Treatment Reviews, Journal of Clinical Epidemiology, PLOS ONE, Journal of Translational Medicine, BMC Medical Research Methodology, Clinical Trials, Journal of Empirical Research in Human Research Ethics, Journal of Evaluation in Clinical Practice, Open Medicine, Biomarker Research, Human Genomics and Proteomics, Research Synthesis Methods, editor-in-chief, European Journal of Clinical Investigation)
Prof Rich Ivry, University of California, Berkeley, US (editorial boards: Journal of Cognitive Neuroscience, Cerebellum)
Prof Jaakko Kaprio, University of Helsinki, Finland (editorial boards: Addiction, Nicotine & Tobacco Research, Twin Research and Human Genetics)
Prof Keith Laws, University of Hertfordshire, UK (editorial boards: BMC Psychology, PLOS ONE)
Dr Bernard Le Foll, University of Toronto, Canada (editorial boards: Neuropsychopharmacology, PLOS ONE, Nicotine and Tobacco Research, Frontiers in Behavioral and Psychiatric Genetics, Scientific World Journal, Open Addiction Research, Journal of Addiction Research and Therapy, International Journal of Clinical and Experimental Medicine, American Journal of Translational Research)
Prof Steven Luck, University of California, Davis, US (editorial boards: Psychological Science, Attention, Perception & Psychophysics)
Prof Michael Lynskey, Kings College London, UK (editorial board: Addiction)
Dr James MacKillop, University of Georgia, US (editorial boards: Addiction, Nicotine & Tobacco Research)
Prof John McLeod, University of Bristol, UK (editorial board: Addiction)
Dr Tom Manly, MRC Cognition and Brain Sciences Unit, UK (editorial boards: Neuropsychology, Neuropsychological Rehabilitation)
Prof Antony Manstead, Cardiff University, UK (editorial board: Journal of Personality and Social Psychology, former chief editor, British Journal of Social Psychology)
Prof Jason Mattingley, University of Queensland, Australia (editorial boards: Brain & Cognition, Cortex, Cognitive Neuroscience, Neurocase, Neuropsychologia)
Prof Matt McGue, University of Minnesota, US (editorial Boards: Addiction, Behavior Genetics)
Dr Hayden McRobbie, Queen Mary University of London, UK (editorial boards: Addiction, Journal of Smoking Cessation, Nicotine & Tobacco Research)
Prof Marcus Munafo, University of Bristol, UK (editorial boards: Addiction, Drug and Alcohol Dependence, Nicotine & Tobacco Research, Annals of the New York Academy of Sciences: Addiction Reviews, International Journal of Neuropsychopharmacology, International Journal of Molecular Epidemiology and Genetics, Journal of Psychopharmacology)
Dr Rachael Murray, University of Nottingham, UK (editorial board: Addiction)
Dr Ray Niaura, Schroeder Institute for Tobacco Research and Policy Studies, US (editorial boards: Nicotine & Tobacco Research, BMC Public Health, Journal of Addiction, American Journal of Health Behavior)
Dr Brian Nosek, University of Virginia, US (guest editor for the Registered Replications project, Social Psychology)
Prof Brad Postle, University of Wisconsin-Madison, US (editorial board: Journal of Cognitive Neuroscience, former handling editor, Neuroimage)
Prof Wim Riedel, Maastricht University, Netherlands (editorial boards: Journal of Psychopharmacology, International Journal of Tryptophan Research)
Prof John Rothwell, University College London, UK (editorial boards: Brain Stimulation, Neurorehabilitation and Neural Repair, editor-in-chief, Experimental Brain Research, former deputy editor, Brain)
Dr Pia Rotshtein, University of Birmingham, UK (editorial board: Cortex)
Dr Ayse Saygin, University of California San Diego, US (editorial boards: PLOS ONE, Frontiers in Human Neuroscience)
Dr Peter Selby, University of Toronto, Canada (editorial board: Addiction)
Dr Lion Shahab, University College London, UK (editorial board: Addiction)
Dr Jon Simons, University of Cambridge, UK (editorial board: Quarterly Journal of Experimental Psychology)
Prof Barbara Spellman, University of Virginia, US (editor, Perspectives on Psychological Science)
Dr Petroc Sumner, Cardiff University, UK (editorial board: Journal of Experimental Psychology: Human Perception & Performance)
Dr Peter Talbot, University of Manchester, UK (editorial Board: Journal of Psychopharmacology)
Dr Nicholas Timpson, University of Bristol, UK (editorial board: PLOS ONE)
Dr Michael Ussher, St George's, University of London, UK (editorial board: Addiction)
Prof Wim van den Brink, University of Amsterdam, Netherlands (editorial boards: Sucht, Suchttherapie, editor, European Addiction Research)
Prof Frederick Verbruggen, University of Exeter, UK (editorial boards: Experimental Psychology, Quarterly Journal of Experimental Psychology)
Dr Antonio Verdejo-Garcia, Monash University, Australia (editorial boards: Addiction, PLOS ONE)
Dr Ed Vul, University of California, San Diego, USA (editorial board: PLOS ONE)
Prof Eric-Jan Wagenmakers, University of Amsterdam, Netherlands (editorial boards: Cognitive Psychology, Journal of Mathematical Psychology, Psychonomic Bulletin & Review)
Prof Andrew Whitehouse, University of Western Australia (editorial Boards: Journal of Autism and Developmental Disorders, Journal of Speech, Language and Hearing Research, PLOS ONE)
Dr Jelte Wicherts, Tilburg University, Netherlands (editorial boards: Intelligence, Psychological Methods)
Prof Reinout Wiers, University of Amsterdam, Netherlands (editorial boards: Addiction, Frontiers in Psychopathology, Addiction Biology)
Prof Jeremy Wolfe, Harvard University, US (editor-in-chief, Attention, Perception, & Psychophysics)
The opinions expressed above are those of the signatories and need not represent the official views of the named journals.
View the original article here
A poll shows parents are stopping their children playing outside due to fear of upsetting their neighbours. Photograph: Alamy
What's needed to raise the visibility and representation of women scientists in the UK? Photograph: Peter Macdiarmid/Getty Images