AI-based prediction of academic success: support for many, disadvantage for some? - Herrmann, Lisa; Weigert, Jonas (2024)
The use of computational tools to predict academic success has become increasingly popular. Machine learning algorithms, trained on past study histories, have been shown to provide valid predictions. However, knowing about biases and unfairness in algorithms, one should take a closer look at these predictions. This paper explores the extent to which the predictive accuracy of academic success varies between specific groups of students, focusing on traditional and non-traditional students (NTS), who have not acquired a higher education entrance qualification at school. In a case study the study compares several popular algorithms and their prediction quality, and investigates whether misclassified NTS show positive or negative biases. Results revealed that the accuracy of predicting academic success for NTS was significantly lower than when considering all students as a whole. The direction of the distortion cannot be determined exactly due to small case numbers. The study emphasizes that the possibility of bias always has to be considered when predicting study success, and the use of such tools must ensure there are no undesirable biases that could affect certain student

Data-driven Student Advisory and Potential Direct Discrimination: A Literature Review on Machine Learning for Predicting Students' Academic Success - Schömer, Daniel; Laumer, Sven; Treischl, Edgar; Weigert, Jonas; Wilbers, Karl; Wolbring, Tobias (Americas Conference on Information Systems, 2021)
The use of machine learning found its way into the field of educational research and address the necessity of data-driven student advisory. Notwithstanding the plethora of current research that predicts and quantifies aspects of students' academic success in various situations during a student's course of study, most studies focus on the technical aspects and overlook potential of direct discrimination. In addressing this gap, this study systematically reviews and identifies scholarly papers that use sensitive attributes to predict students' academic success, which are not allowed to be considered in the decision-making process due to legal restrictions. The review shows that in 95 studies, over 52 percent use at least one discriminating attribute defined by the International Covenant on Civil and Political Rights. This sheer volume highlights the need for practitioners and scholars to explore causal mechanisms to prevent potential discrimination or seek alternative solutions without using protected individual characteristics. <p
Keywords: Diskriminierung

Design eines Academic Analytics Systems zur Unterstützung des Qualitätsmanagements an Hochschulen - Weigert, Jonas (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2022)
In den letzten Jahren lässt sich der Trend erkennen, dass immer mehr Hochschulen in Deutschland interne Qualitätsmanagementsysteme im Bereich Studium und Lehre aufbauen. Gleichzeitig findet insbesondere im Rahmen von Learning Analytics das Thema Datenanalyse Einzug in die wissenschaftliche Diskussion und praktische Ausgestaltung im Kontext Hoch- schule. Während die Disziplinen jeweils auf ein gefestigtes theoretisches Fundament blicken können, stehen die Arbeiten zur Entwicklung von umfassenden, die Forschungsbereiche ver- bindenden Academic Analytics Systemen noch am Anfang. Der Logik des Design Science Research folgend, wird in der vorliegenden Arbeit im Rahmen eines mehrzyklischen Design Prozesses ein Academic Analytics System entwickelt und in ei- nem konkreten Kontext erprobt. Die Umsetzung erfolgt dabei an einem wirtschafts- und sozi- alwissenschaftlichen Fachbereich. Die Evaluationen werden mit Hilfe der Verantwortlichen der Studiengänge, sowie über technischen Experimente durchgeführt. Auf einer höheren Abstrak- tionsebene wird ein theoretisches Modell zur Gestaltung von Academic Analytics Systemen generiert und im Rahmen der Zyklen evaluiert. Die Arbeit bietet zum einen präskriptive, auf die Gestaltung von Academic Analytics Systemen ausgerichtete Erkenntnisse. So lassen sich auf Basis des entwickelten Modells mögliche Da- tenbasen, Speicher-, Analyse- und Visualisierungsmöglichkeiten sowie Stakeholdergruppen abgrenzen. Darüber hinaus können deskriptive, die theoretischen Grundlagen erweiternde Er- kenntnisse abgeleitet werden. Diese beziehen sich insbesondere auf die Verbindung der un- terschiedlichen Ansätze der Datenanalyse an Hochschulen und deren Bezug zum Qualitäts- management im Bereich Studium und Lehre sowie auf die Einführung neuer in diesem Rah- men relevant werdender Rollen. Zudem lässt sich erkennen, dass die Prognosen des Studien- erfolgs auf Basis unterschiedlicher maschineller Lernalgorithmen bereits früh im Studium mit ansetzen können und eine hohe Genauigkeit aufweisen. Gleichzeitig macht die Arbeit weitere Forschungsbedarfe im Bereich Academic Intelligence deutlic
Keywords: Qualitätsmanagement an Hochschulen; Business Intelligence; Datenana- lyse an Hochschulen; Academic Analytics; Academic Intelligence; Design Science Research

Give a Little, Take a Little? A Factorial Survey Experiment on Students’ Willingness to Use an AI-based Advisory System and to Share Data - Treischl, Edgar; Laumer, Sven; Schömer, Daniel; Weigert, Jonas; Wilbers, Karl; Wolbring, Tobias (Sozialwissenschaftliche Datenerhebung im digitalen Zeitalter, 2021)
Given its ability to handle a large amount of data, artificial intelligence (AI) has the potential to improve data-driven decisions under various situations. The present research identifies the necessary conditions for the implementation of an AI-based advisory system (AS) in higher education. Using a factorial survey design, we examine experimentally varied features of an AI-based AS to explore students’ willingness to use it and students’ willingness to share their data as a core challenge for successful implementation. Theoretically, we focus on the perceived costs and benefits to explain students’ intention, but we also highlight the role of trust and privacy concerns in regard to collecting data for the AS. In terms of benefits, information about the predictive power of the AS significantly increases students’ intention to use the tool and to share data and thus offers an incentive for students to share data. Moreover, a disproportionately long survey duration and survey topics that seem unrelated to the AS reduce students’ willingness to share data. With respect to trust and privacy concerns, our results indicate that providing transparent information about the AS has no effect on students’ willingness to share data, while aspects regarding who has access to the AS results and a long period of data storage reduce students’ intentions to share data. Based on these findings, we advise universities to communicate students’ expected benefits from a system to implement the AS, but we also recommend seriously considering students’ privacy concerns. Who has access to the data and the results of the AS should be transparent, as well as for what reason and how long. Otherwise, a substantial and probably selective part of the student body may not use the tool or share data due to privacy concerns. <

Unterstützung der Studiengangsentwicklung: Konzeption und Erfahrungen zur Studiengangsmatrix - Renner, Moritz; Simon, Carolin; Weigert, Jonas; Wilbers, Karl; Wittmann, Maria (Handbuch Qualität in Studium, Lehre und Forschung, 2019)
Keywords: Studiengangsentwicklung; Qualitätsmanagement; Hochschule; Universität; Studiengangsmatrix