An openness framework for ex ante evaluation of societal impact of research

Abstract

Over the last decade, the idea of societal impact resulting from publicly funded research has changed from being a relatively fringe concern related to high-technology entrepreneurship and spin-off companies to becoming an increasingly important public policy concern. This has stimulated academic science policy research to investigate the impact of research and conceptualize how knowledge created in academic contexts can be coupled to real-world problems or needs. Most of the work in this stream of research focuses on ex post impacts, that is, the impacts created by individual research activities or research strands after their completion and likewise there has been much research on ex post impact evaluation. However, ex ante impact evaluations have become increasingly important for funding decisions, but little is known about how to evaluate impact when considering research project proposals. In this article, we propose a conceptual framework to evaluate ex ante impact of research based on the idea that a research proposal is a ‘promise’ to achieve impact. We suggest that evaluators could assess social impact promises by considering two elements, namely the activities that couple their knowledge with non-academic users and the interdependency and consistency of such activities throughout the overall project proposal. We ultimately propose an analytical framework for refining our ‘openness’ conceptual framework in future empirical research.

Publication
Research Evaluation

Highlights

  1. We reflect on the ex ante impact in research proposals to propose a conceptual framework to assess it.
  2. We conceptualize a research proposal as a promise of what will be done in the future and contend that the credibility of this promise must be assessed.
  3. We draw on the openness framework to approach the propensity of research to create societal impact, as open research practices facilitate incorporating non-academic knowledge into research activities (coupling).
  4. We identify coupling, interdependency, and consistency as key elements of the research proposal to assess the societal impact promises of the proposal.
  5. We propose an analytical framework to assist evaluators with a common criterion to assess the credibility of the potential societal impact of research proposals.