Inspiration or trap? Should you be tracking the impact of your research?


By Prof Mark Reed @profmarkreed



Why is everyone racing to try and track research impact?

Impact tracking: inspiration or trap

Impact tracking: inspiration or trap

Investing in research is critical for maintaining a country’s economic advantage and delivering improvements in human social and environmental wellbeing. Currently, global estimates suggest that countries invest in more than $1.4 trillion in research and development every year. To justify and help target such investment, there is a rapid and growing interest in evaluating the societal and economic impacts of research.

As a result, research funders around the world now increasingly demand that researchers evaluate the likely impacts that their research may have, and ask them to articulate clear “pathways” to achieve those impacts when they apply for funding. Delivering impacts from research is increasingly important in assessments of research quality (such as the Research Excellence Framework (REF) in the UK and Germany’s Research Rating (Forschungsrating) system). Such assessments now consider both research excellence and the extent to which the research has had ‘reach and significance’ in terms of its impacts on economy, society and/or culture.

Partly the goal of these assessments is to monitor and celebrate impacts, so that future public funding for research can be more easily justified. Partly, these assessments also aim to incentivise change within the academy, and as financial rewards follow positive assessments of impact, these incentives are increasingly trickling down into annual performance reviews and promotions exercises within the institutions that are being evaluated. These changes have had a mixed reception.

The research impact trap: are we playing self-defeating games with impact?

There is evidence that previous assessments of research quality in the UK have influenced the kinds of funding available and the nature of research conducted (for example, see these examples from Geography by Sidaway (1997), Minca (2000), Shelton et al. (2001), Short (2002), Berg (2004) and Birnie (2005)). As yet, however, there has been no assessment of the effects of the recent move to auditing the impact of research. Concerns have been expressed about the opportunity costs of evaluating impact and the negative impacts that this may have on academic freedom, which is it argued may “retrench an elite model of power/knowledge relationships”.

Concerns have also been expressed about how these evaluations might affect the types of impact that are pursued by academics. Impact evaluation criteria may constrain and shape perceptions of “what counts as impact” and funnel future effort into narrowly conceived types of impact generation activities. As a result, certain outcomes described as impacts while other important outcomes from research are omitted from narratives about the impact of research. And it is not clear whether the perceptions of academics about the impact of their work always match those of the stakeholders involved in or benefiting from the research.

Impact monitoring: bottled inspiration

Impact monitoring: bottled inspiration

What is clear is that different people are racing to track impact for very different reasons. Individual researchers may be motivated extrinsically to demonstrate impact so that they can get on in their career. They may be intrinsically motivated to see if they are achieving their personal goals, so they can learn how to make more of a difference. Research and impact managers in institutions are trying to understand how they can target help to researchers who have the potential to make a difference, so that their institution can collectively demonstrate greater impact. Much of this is extrinsically motivated by the need to achieve high scores in impact evaluations that will feed into rankings, reputation and financial rewards for the institution. Although much of this might feel like game-playing, and probably is, I have yet to meet a knowledge exchange professional in an impact-orientated role who gets their kicks from playing the game. Ultimately, our goal collectively is to help each other raise our game to make our research more relevant and change more lives for the better. If we can change the narrative from bean-counting to progress-watching, then we might also change the hearts of those who are extrinsically motivated by the impact agenda, so see that the impacts they seek, go far beyond promotions and rankings. Tracking impact might become more about sharing good practice and inspiring each other to go the extra mile. After all, the impacts we leave from our research are our true legacy. And when we look back on our careers, that legacy won’t be measured in the number of points our impacts scored in a Government assessment; it will be measured in lives changed.

Look at yourself

So why are you reading this blog? Why are you interested in tracking impact? There are probably a whole mix of reasons, including egotistical and extrinsic motives, as well as more altruistic and intrinsic motives. But whatever your motives might be, I encourage you to look at something you’ve done that has had a positive impact on the world around you, no matter how small that may be. Look hard at what you did, and now look at how you felt at the time, and how you feel now.

If there is even the slightest spark of inspiration there, bottle it, because that is the most powerful thing you can take from evaluating your impact. Focus on that inspiration and make it bigger and brighter, and think how you can build on it. Think about how you can share that spark of inspiration with others, so that they are inspired to think differently about their research. It might be such a small impact that you think no-one would ever want to report it an any formal evaluation of research impact. But that’s not the point: if we focus on the one first, small impact we can make this month, and keep thinking how we can make it bigger, someone somewhere will one day be interested enough to report it.

In their book, The One Thing, Gary Keller and Jay Papasan use the metaphor of dominos to explain this. A domino is able to knock down another domino that is 50% larger, so a 2-inch domino is able to knock down a 3-inch domino. If you extrapolate this, after a run of 8 dominos, your 2-inch domino is able to knock over one the size of a door. In theory, a run of 31 dominos could knock over a domino taller than Mount Everest. I encourage you today to think of one thing you could do this month that would create impact from your research – and start working on it today. Don’t think of something you could do if you had the time or resources, or that you might do tomorrow. Think of something you can actually do today that will start you on your pathway to impact.