Reporting standards to enhance article reproducibility
Beginning May 1st Nature Methods will be requiring authors of manuscripts being sent back to peer review to fill out a checklist to disclose technical and statistical information about their submission.
The May Editorial briefly describes why we are using this checklist and provides some details of what is included. Authors can find the checklist that Nature Methods will be using at http://www.nature.com/nmeth/pdf/sm_checklist.pdf and there is a link to it on the journal homepage. Our checklist is identical to that of most of the other Nature journals except for an added item asking authors to “Identify all custom software or scripts that were required to implement the methodology being described and where in the procedures each was used.” Based on feedback we have received, a missing software or script seems to be the item most often mentioned by people commenting on challenges in reproducing a method we have published. This reporting requirment is an important step in trying to address this deficiency.
We expect that the addition of these reporting requirements will elict some grumbling by authors. But based on the experience of Nature Neuroscience, which has been requiring authors to fill out a methods checklist before even the first round of review, we expect authors will come to appreciate the role it serves.
The checklist is only one part of the efforts the Nature journals are making to improve reproducibility. The other journals are also removing formal limits on the length of the methods section. But since Nature Methods has long had no limits on the length of our Methods section, the checklist is the most prominant change for us and our authors.
The May issue also contains other articles relevant to reproducibility. The Correspondence section has a discussion about analyzing the reproducibility of animal experiments. And the May Technology Feature discusses reproducibility in quantitative PCR, a methodology that has suffered from serious problems in this regard due to poor experimental technique and reporting.
For those not tired of reproducibility at this point Nature also has a Special Focus on Challenges in irreproducible research.
As has been said in the editorials on the subject, this is only a first step toward improving the reproducibility of our published research and we welcome feedback from the community on our efforts.