What is meaningful research and how should we measure it?

Helmer S, Blumenthal DB, Paschen K (2020)


Publication Type: Journal article

Publication year: 2020

Journal

Book Volume: 125

Pages Range: 153-169

Journal Issue: 1

DOI: 10.1007/s11192-020-03649-5

Abstract

We discuss the trend towards using quantitative metrics for evaluating research. We claim that, rather than promoting meaningful research, purely metric-based research evaluation schemes potentially lead to a dystopian academic reality, leaving no space for creativity and intellectual initiative. After sketching what the future could look like if quantitative metrics are allowed to proliferate, we provide a more detailed discussion on why research is so difficult to evaluate and outline approaches for avoiding such a situation. In particular, we characterize meaningful research as an essentially contested concept and argue that quantitative metrics should always be accompanied by operationalized instructions for their proper use and continuously evaluated via feedback loops. Additionally, we analyze a dataset containing information about computer science publications and their citation history and indicate how quantitative metrics could potentially be calibrated via alternative evaluation methods such as test of time awards. Finally, we argue that, instead of over-relying on indicators, research environments should primarily be based on trust and personal responsibility.

Authors with CRIS profile

Involved external institutions

How to cite

APA:

Helmer, S., Blumenthal, D.B., & Paschen, K. (2020). What is meaningful research and how should we measure it? Scientometrics, 125(1), 153-169. https://dx.doi.org/10.1007/s11192-020-03649-5

MLA:

Helmer, Sven, David B. Blumenthal, and Kathrin Paschen. "What is meaningful research and how should we measure it?" Scientometrics 125.1 (2020): 153-169.

BibTeX: Download