Development of a Standardized Rating Tool for Drug Alerts to Reduce Information Overload

Pfistermeister B, Sedlmayr B, Patapovas A, Suttner G, Tektas O, Tarkhov A, Kornhuber J, Fromm M, Buerkle T, Prokosch HU, Maas R (2016)

Publication Type: Journal article

Publication year: 2016

Journal

Methods of Information in Medicine Thieme Publishing

Book Volume: 55

Pages Range: 507-515

Journal Issue: 6

DOI: 10.3414/ME16-01-0003

Abstract

A well-known problem in current clinical decision support systems (CDSS) is the high number of alerts, which are often medically incorrect or irrelevant. This may lead to the so-called alert fatigue, an overriding of alerts, including those that are clinically relevant, and underuse of CDSS in general.The aim of our study was to develop and to apply a standardized tool that allows its users to evaluate the quality of system-generated drug alerts. The users' ratings can subsequently be used to derive recommendations for developing a filter function to reduce irrelevant alerts.We developed a rating tool for drug alerts and performed a web-based evaluation study that also included a user review of alerts. In this study the following categories were evaluated: "data linked correctly", "medically correct", "action required", "medication change", "critical alert", "information gained" and "show again". For this purpose, 20 anonymized clinical cases were randomly selected and displayed in our customized CDSS research prototype, which used the summary of product characteristics (SPC) for alert generation. All the alerts that were provided were evaluated by 13 physicians. The users' ratings were used to derive a filtering algorithm to reduce overalerting.In total, our CDSS research prototype generated 399 alerts. In 98 % of all alerts, medication data were rated as linked correctly to drug information; in 93 %, the alerts were assessed as "medically correct"; 19.5 % of all alerts were rated as "show again". The interrater-agreement was, on average, 68.4 %. After the application of our filtering algorithm, the rate of alerts that should be shown again decreased to 14.8 %.The new standardized rating tool supports a standardized feedback of user-perceived clinical relevance of CDSS alerts. Overall, the results indicated that physicians may consider the majority of alerts formally correct but clinically irrelevant and override them. Filtering may help to reduce overalerting and increase the specificity of a CDSS.

Authors with CRIS profile

Barbara Pfistermeister Lehrstuhl für Klinische Pharmakologie und Klinische Toxikologie Brita Sedlmayr Lehrstuhl für Medizinische Informatik Andrius Patapovas Lehrstuhl für Medizinische Informatik Gerald Suttner Department of Psychiatry and Psychotherapy Johannes Kornhuber Lehrstuhl für Psychiatrie und Psychotherapie Martin Fromm Lehrstuhl für Klinische Pharmakologie und Klinische Toxikologie Hans-Ulrich Prokosch Lehrstuhl für Medizinische Informatik Renke Maas Professur für Arzneimitteltherapiesicherheit und Klinische Pharmakologie

Involved external institutions

Molecular Networks GmbH DE Germany (DE) Universität Bern CH Switzerland (CH)

How to cite

APA:

Pfistermeister, B., Sedlmayr, B., Patapovas, A., Suttner, G., Tektas, O., Tarkhov, A.,... Maas, R. (2016). Development of a Standardized Rating Tool for Drug Alerts to Reduce Information Overload. Methods of Information in Medicine, 55(6), 507-515. https://doi.org/10.3414/ME16-01-0003

MLA:

Pfistermeister, Barbara, et al. "Development of a Standardized Rating Tool for Drug Alerts to Reduce Information Overload." Methods of Information in Medicine 55.6 (2016): 507-515.

BibTeX: Download