2 November 2009
In a letter published in this week's Nature, UvA psychologists Jelte Wicherts and Marjan Bakker criticise revised guidelines that make it more difficult to check the results of analyses presented in psychology publications. Their letter was written in response to the revised version of the Publication Manual drawn up by the American Psychological Association (APA).
Scientific research findings must be verifiable. While true in theory, researchers are not always so forthcoming with data in practice. Despite the obligation on psychologists publishing in APA journals to make their research data available in order to facilitate verification of the statistical analyses, it turns out that more than 70% of psychologists neglect to do so.
The result is that the - error-prone - statistical analyses in psychological studies are often impossible to verify. Indeed, unlike journals in other disciplines, psychology journals do not require authors to include an appendix with study data in their published research articles.
The APA's revised Publication Manual lays down clearer rules for sharing research data. According to Wicherts and Bakker, however, these new guidelines actually only promote the rights of the original researcher, thus rendering critical reanalyses all but impossible.
The new APA guidelines stipulate that data can only be shared after the original researcher and the party requesting the data have come to a written agreement. Amongst other things, the agreement sets out the aim of the reanalysis, expectations as regards co-authorship in any resulting publications and restrictions on the publication of findings from the reanalysis. The guideline provides that a researcher whose analysis is questioned can refuse to allow the requesting party to report on the data in an article unless he or she is named as a co-author. But, say Wicherts and Bakker, the majority of researchers will not soon agree to publish an article proving that their original analyses were faulty.