Unpublished research must see the light of day

This is the first in a series of short pieces about open science, in which Michael Sanders describes the findings of a series of papers that have been languishing without being released, as he releases them over the next several months.

The Chinese use two brush strokes to write the word ‘crisis.’ One brush stroke stands for danger; the other for opportunity. In a crisis, be aware of the danger – but recognize the opportunity.

President JF Kennedy

You might be forgiven for thinking that now is an especially tough time for social science. The reproducibility crisis that has engulfed much of social psychology, and threatens most of the rest of empirical social science, has called many findings, previously viewed as cast iron, into question. Economics has not had a great decade since the financial crisis, which put its popularity and influence perhaps at its lowest ebb to date. The use of machine learning and other technically elaborate approaches to cure social ills has not lived up to the much hoped for promise.

But, as President Kennedy told us, we should be aware of the danger – but recognise the opportunity.

The changes in empirical social science over the last 10 or 20 years have mostly been for the better. We have better quality data, which is more widely accessible, and easier to analyse, than ever before. The crisis at the heart of psychology has brought about a generation of researchers committed to open science, and has changed the norms in the profession. Pre-registration is now commonplace, publishing of data and code is on the rise. This isn’t as dramatic or newsworthy as a public fight over “power posing”, but it’s much more important.

For all these improvements, there is one area where open science has yet to have the same level of success, and that’s in publication. Publication bias – whereby interesting, surprising, or “positive” findings are more likely to be published than more quotidian results – is pervasive, both because of the views of editors and reviewers at peer-reviewed journals, and because of the career incentives for researchers, who need to make a name for themselves, usually in the first few years of their careers, and get their papers published and cited heavily – something that discourages investment in papers that lack “originality” and which have already been rejected by the handful of top journals in a given field (economics is particularly bad for this).

Looking at my own research to date, too much of it remains unpublished. In some ways this is understandable – I am an academic researcher one day a week, and my incentives are not skewed towards investing in older papers that have already been rejected a couple of times. But, I am a believer in, and campaigner for, open science – and I need to practice what I preach. I’ve identified 15 papers that are either finished or close to finished, and which I’ve been sitting on for more than 18 months. This isn’t good enough.

So, this is the first blog in a series. I’ll be publishing each of these papers as working papers over the coming months, each accompanied by a short blog. The papers won’t have gone through peer review when I release them, so they should be taken with a pinch of salt – but they’ll be out in the world for you to judge them, and maybe learn something interesting.

Dr Michael Sanders is Chief Executive of What Works for Children’s Social Care, and a Reader in Public Policy at the Policy Institute, King’s College London.