What do we actually do when we do impact?

A long time ago, in a galaxy far, far away (well, when we had a UK Government that was thinking about localism and “The Big Society”) the Arts and Humanities Research Council’s Connected Communities programme funded three projects, along with the UK Department for Communities and Local Government (DCLG), to see how the research the programme had invested in so far could help policy. Skip forward a year and the teams involved in doing these reviews concluded that they had not, exactly, gone to plan. So, I ended up joining them on a project called Translation Across Borders to try and find out why.

Well, I have a paper out based on this project in Evidence & Policy. I’ll attempt to summarise it here.

Now, there is absolutely oodles of research out there, across numerous disciplines, on how and why policy-makers use evidence in their decisions, and the barriers to this. The unique value-added of this project was that it was co-produced with a civil servant who was actually involved in policy-making. Our co-author, Robert Rutherfoord, is a Principal Social Research at DCLG, and did fieldwork with me.

My role was to interview all the academics who had participated in producing the original policy reviews, with Robert, and find out what they had done and the barriers they found in taking their evidence into a policy-making environment. Our literature review found that doing this is remarkably rare – us academics seem to love asking policy-makers what they think the barriers are, and how they use evidence, but we don’t ask us academics what we think the barriers are. This is all the more surprising given all the wailing and gnashing of teeth regarding the Research Excellence Framework’s measurement of socio-economic impact since 2013.

What did we find? In the interpretive approach we took to analysing the data, three things stood out. Firstly, as academics, we construct our identities as biographies (like everyone else on the planet). These are key meaning-making devices for us and help situate us, and our practices in the here-and-now. Secondly, these biographies are strongly linked to disciplinary identities. Unsurprisingly, some disciplines – like policy studies – more commonly do work with policy-makers, or attempt to affect change in policy, than other disciplines. This is a bit of a “no shit, Sherlock” finding, but surprisingly it is not dealt with a lot in the literature, perhaps because the need for diverse disciplines to affect policy-making has only emerged in the last decade and they are only just beginning to self-reflect. On this count, I find the delightfully naïve debates in mainstream political science interesting when you compare them to policy studies, who have been concerning themselves with this issue for the last 70 years. The final insight was that institutional pressures, particularly the demand to produce 4* journal articles for the REF means that the sorts of activities that are recognised to help deliver “impact” – developing working relationships with policy-makers and networks of influence – are not prioritised or encouraged within internal performance management systems.

Now, a lot of this will come as no surprise to many academics. Indeed it didn’t necessarily come as a surprise to us. What did come as a surprise to us, and why this research is important, was that this our civil servants we were co-producing with did not know about much of this, particularly things like the impact of the REF on behaviour and incentive structures. Therefore, our recommendation as to what should be done better is a bit different to most other similar projects. Whereas a lot of “toolkits” and other training focused on getting academic evidence into policy-making focuses on “knowing your audience”, from a variety of different perspectives, we instead focused on the need for academics to know themselves better. Because, basically, academics are weird. We behave in a lot of ways that are completely alien to those outwith academia. And we need to pause and think about this every now and then. And also, policy-makers who want to work with academics would do well just to spend a short amount of time learning about what makes them tick, and understanding that there is diversity in what academics do, and how they do it.

To this end we did create some tools from this project to try and make this process a bit easier. One of these is some fun “academic archetype” cards that can be used to prompt reflection, and also help policy-makers understand academics a bit better. If you want to use these, please drop me an email, and this can be arranged. I’ll be presenting them at a “Research Bite” seminar in the University of Stirling Library Enterprise Zone on 2 August at 12:30. I’ll also probably bring them out at a session at the Australian National University on 11 October at 13:00, and possibly when I’m at the Department of Social Policy and Intervention, University of Oxford from the 13 November for a week.

We’re just organising Gold OA or Green OA for the paper, but in the meantime drop me an email if you want a copy and you don’t subscribe to E&P.