Qualitative Indicators

What kind of qualitative indicators can we use to measure research impact? The most powerful tool to describe your impact is often a narrative (impact story), enhanced with quantitative indicators.

Overview

When talking about qualitative indicators used to measure research, we often refer to indicators that measure the quality of researchers' results, such as publications.

There are several different measures to evaluate the quality of publications but the most commonly known in Finland is the Publication forum system (JuFo). Publication Forum system is based on quality classification of scientific publication channels. Rating takes place in field specific expert panels. Evaluations based on bibliometric methods can be complemented with JuFo which is suitable for evaluation of publications of research institutes, disciplines or countries but not for evaluation of individual researchers. JuFo -levels of publications are used as one criteria in the funding model for universities by the Ministry of Culture and Education.

Rating publications may tell you something about the scientific impact of research but nothing about the actual societal or economic impact.

Journals can in theory be evaluated in two different ways: with subjective assessment and with quantitative metrics. Usually researchers know very well which journals have the most prestige value in their field. Rating publications may tell you something about the scientific impact of research but nothing about the actual societal or economic impact. Nowadays also the openness of publications counts.

Funders such as the Academy of Finland and the EU require that research projects publish their results via various modes of open access to secure that research results reach the academic community quickly .

Impact story

When measuring societal or other impacts of research, qualitative indicators can also include interviews, observations and corroborations of third parties that express the perceptions of the changes that have occurred during and after a research project. Often the most effective qualitative tool to verify your research impact is a narrative (case study/impact story) describing the effect your research has had on the society, economy or culture. It allows you to critically assess how you succeeded in fulfilling the aims you had regarding research impact.

Recently impact stories have been used also in research evaluation, for example, in the UK as part of the Research Excellent Framework 2014. Of course, stories can and should, when possible, be supported with data and quantitative indicators. Impact stories very often present the results in favorable light, nevertheless they do give valuable insight about the results of the research.

An effective impact story includes: 

  • A short summary of the research and the impact
  • Description of the conducted research (what was done, where and by whom)
  • Description of the impact (What happened and when? What and where was the change and why and to whom did it matter?)
  • Description of the process that lead to impact (independent, verifiable evidence. ) Add quantitative indicators to stenghten your case.

(Source: LSE Impact Story guidance).

The Strategic Research Council in Finland uses impact stories in reporting the results of the SRC funded projects. They have published a very comprehensive template for impact stories here (in finnish). Impact stories are also a great tool to communicate your research to external stakeholders and society at large. Check out for example University of Oulu's project Bright Clouds Dark Clouds (funded by SRC) stories on their website: BCDC impact narratives.

Create your impact plan!

You can now create your own impact plan. Our template provides you with guiding questions to help map out your impact and helps you link your research with sustainable development goals. Just click here!