A couple of weeks ago I submitted a commentary proposal for Zwaan, Etz, Lucas, & Donnellan’s excellent BBS article: Making Replication Mainstream. It wasn’t about war (obviously), but I was on holidays last week and the proposal got rejected today, so… well, I don’t have much else to say, but I’d love to hear what you think about it!
Proposal: Zwaan et al. raise a number of concerns about replications, address these concerns, and conclude that “there are no theoretical or statistical obstacles to making direct replications a routine aspect of psychological science”. However, the concerns addressed are all (negative) attitudes and beliefs about replications: e.g. “replications are a distraction”, “replications affect reputations”, and so on. Theories of behaviour (change) certainly include a role for such attitudes in influencing the behaviour in question, but also emphasise the importance of self-efficacy and control (e.g. Ajzen, 2002). I have conducted a study on social psychologists’ beliefs about reformative research practices, including replications (data and write up available here: https://osf.io/hjxkq/). A re-analysis of these data specifically with Zwaan et al. in mind, shows that beliefs and attitudes (e.g. “The focus on direct replications is happening at the expense of other important issues in social psychology”, “The proportion of findings in social psychology that do not directly replicate is unacceptably high”) predict self-reported likelihood of engaging in replication projects. However, a sense of self-efficacy (e.g. “As an individual researcher, I can effect change in the academic field of psychology”) also has predictive power. Thus, there are both theoretical and empirical reasons to suspect that even as Zwaan et al. convincingly addressed the “theoretical or statistical obstacles” to replications becoming mainstream, there are some practical obstacles left.
Relevant Expertise: I have participated in the replication debate on social media (e.g. see blog here) and am participating in a panel discussion at SPSP 2018 on “Introduction to Open Science: Nuts and Bolts of Getting Started”. The two short papers I wrote based on a survey of psychologists’ responses to the replication crisis have been downloaded from the OSF a combined total of 450 times. While I am highly positively inclined towards (direct) replications, I have not completed any of my own; thus I am personally aware that attitudes are not the only thing that influences the likelihood that replications will become mainstream.