Beyond indicators, what is the social impact of research?

Many research organizations share the goal of social impact. However, it is not always easy to find definitions and frameworks that help understand what social impact is, how it is integrated into the research process and how it can be evaluated. In this post we want to share the learnings from a reflection process carried out at Orkestra, with the aim of assisting other people and organizations that are facing similar challenges. In order to better respond to this objective, throughout the post we talk about social impact, which is the concept that best allows us to link our reflection to that of other research institutes and universities. However, it seems relevant to clarify that in the case of Orkestra there is another concept, impact on the territory, which better represents our work as it brings together impacts in the economic, social, political and environmental spheres. 

What is social impact?

In the academic world there is a lot of talk about academic impact, or in other words, publishing new knowledge in impact journals (redundancy aside) so that, through them and in the long term, changes occur in society. We have very clear indicators that measure how much we publish in these journals, and yet it is practically impossible to measure the change in society that these journals generate. This is an example of a way of understanding impact in which knowledge is first generated and then translated into a knowledge product (report, presentation, article) and then the researcher interacts with society seeking to have a social impact.

But there are research processes that do not prioritize academic impact as a path to social impact, but seek to generate social impact during the process of knowledge generation. This second type of social impact, as opposed to the classic model of impact generation, does not occur following a publication that gathers the new knowledge generated, but rather during the process of co-generating new knowledge with actors in the territory within the framework of their policies. This social impact is also difficult to measure in quantitative terms, but it is much easier to describe than that generated by journals, because researchers participate directly in the impact and both our voices and those of the actors with whom we work can tell the story of the impact. In this line, we are exploring methodologies and narratives, participatory evaluations and other qualitative approaches that allow us to capture more systematically the value generated for the actors of the territory in the research process.

In theory, academic and social impact are not only compatible, but interdependent, and it is desirable to constructively combine the two. In practice, however, this relationship is complex and it is not easy to be constructive. At Orkestra we have the ambition to generate a mutually enriching relationship between these two perspectives, and this post shares some reflections and lessons learned along the way. 

A critical reflection on the linearity of impact

We refer to the linearity of the value chain of social impact generation according to the following process: the academy generates knowledge, which is published, and then transferred to the actors of the territory so that they can apply or implement it. When these actors apply the research to transform their reality, we consider that it has generated social impact.

However, the linearity of research has been questioned in connection with some concepts that have been investigated at the institute, such as citizen science, action research, some forms of collaborative governance, and Responsible Research and Innovation (RRI), which is a concept promoted mainly in Europe, and especially within the Horizon 2020 program.

The central idea shared by these approaches is to open research, innovation and decision-making processes to the active and deliberate participation of society in order to align academic results with societal values, needs and expectations. Most of our research processes are carried out with policy makers, either from the Basque Government, regional governments, city councils and their agencies or representatives of organizations linked to business competitiveness. Although less frequently, research processes have also begun to be opened to other groups such as young people or people at risk of exclusion. These actors have participated in the knowledge generation processes and, naturally, while co-generating knowledge, they have been integrating it into their practices, for example, through the transformation of their governance models or the definition of new policies. In other words, the impact is bidirectional (research impacts the actors' practices and, in turn, practice impacts research).  In this case, the social impact has been produced during the dialogue and interaction between researchers and stakeholders in the territory, and not through publications that come about once the process has been completed.  Later, if the learning generated is published, another process is opened that can have an impact on other people and communities. 

Consequently, one of the debates that has opened up around these new ways of understanding research is that of social impact. What is the impact that research should have on society? What is its current impact? How is impact produced? How can it be measured or communicated?

Some answers we take from the experience of other organizations

The above are questions to which some answers can be found in the literature, and to which other organizations have already responded. The first step has been to learn from what is already being carried out by others, and this path has been taken by Orkestra's communication team, looking for how other research organizations are approaching the issue⁽¹⁾. One of the lessons learned from this search has to do with how the two types of impact mentioned in the introduction have been conceptualized:

1. Academic or scientific impact, which influences the thinking of other researchers through publications, networks, conferences, etc.
2. Social impact, which, according to these approaches, occurs when the scientific impact transcends the scientific community and produces changes in society.

It is important to point out that these sources state that academic and social impact are interrelated and are not mutually exclusive, quite the contrary.

There is wide acceptance of this idea that the two types of impact are interrelated. However, there are different ways of understanding this interrelationship. For example, Figure 1 implies that social impact only occurs when scientific impact transcends from academia to society, which implies a linear view in which academic impact occurs first, and then, when it transcends, social impact. We believe that these frameworks, based on the widespread idea that academic impact comes first and then social impact, can be a useful tool for reflection. 

Figure 1. An example of a linear presentation of impact.

Source: Own elaboration

This starting point, based on linear models of impact generation, allows for relevant approaches to research on competitiveness and wellbeing. For example, when a research team, through data analysis, generates new knowledge, which it publishes and makes available to stakeholders in the territory. These actors, either directly through the reading of these documents, or through conferences, seminars, etc., can improve their understanding of a situation, or be inspired by the possible solutions to problems they face. The improvement of the processes of analysis, reflection and decision making of these actors, which then translates into their actions, has an impact on society.

However, we consider that these frameworks are a good starting point for reflection, but they do not cover all the ways in which research is carried out at the Institute, and for this reason, we make a broader interpretation of the outputs, and include the processes here as well.  That is to say, in many of the research processes that we have experienced, this linearity does not occur in the generation of social impact because the concepts described above can be given - and in fact are given - long before obtaining the output that gathers the knowledge. The concepts described below, however, do help us to reflect on the different models for generating social impact in the Institute:  

The need for research institutes to also construct their own answers

We believe that some of the answers to the above questions are contextual, and that, like other research institutes that aspire to generate social impact, Orkestra will have to work out how to adjust them to its mission and the diversity of approaches to research that coexist in the Institute. In this section we address one of these dimensions.

We have pointed out that the scheme presented in Figure 1 does not respond to all forms of research at our Institute, given that the impact generation frameworks found to date are based on classical perceptions of research. However, in many of our research processes the actors of the territory (policy-makers) are not only receivers, but actors of knowledge as well.

A pluralistic perspective, where different methodologies coexist, requires an open approach to impact. A first step, which we propose in this post, is that the process to reach social impact includes the four elements of the previous figure, without predetermining how they are related or interact, so that each project can reflect on this relationship and explain how it is understood by the participating researchers and actors of the territory.

Thus, in a project with a classic and linear perspective, the approach would be as follows:

Figure 2. Impact generation with a classical and linear perspective.

Source: Own elaboration

However, projects with approaches such as action research could propose interrelationships more in line with their methodological principles, for example:

Figure 3. A framework of social impact in cogeneration processes.

Source: Own elaboration

The above reflections open a path in which the definition we give to social impact can lead us to explore innovative ways of understanding our research and the relationship we establish, through it, with the societies in which we live. It is therefore a task that we propose to carry out in the long term and in a participatory manner with all those who work at the Institute and with our collaborating entities.

• (1) http://www.deusto-publicaciones.es/deusto/pdfs/sociallab/sociallab04.pdf
       https://www.helsinki.fi/en/news/education/new-online-course-gives-powerful-tools-increase-research-impact
       https://www.ucd.ie/impacttoolkit/whatisimpact/