Coping with Data Distribution Shifts: XAI-Based Adaptive Learning with SHAP Clustering for Energy Consumption Prediction

Clement T, Nguyen Truong Thanh H, Kemmerzell N, Abdelaal M, Stjelja D (2024)

Publication Type: Conference contribution

Publication year: 2024

Journal

Lecture Notes in Computer Science Springer Verlag

Publisher: Springer Science and Business Media Deutschland GmbH

Book Volume: 14472 LNAI

Pages Range: 147-159

Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Event location: Brisbane, QLD AU

ISBN: 9789819983902

DOI: 10.1007/978-981-99-8391-9_12

Abstract

Adapting to data distribution shifts after training remains a significant challenge within the realm of Artificial Intelligence. This paper presents a refined approach, superior to Automated Hyper Parameter Tuning methods, that effectively detects and learns from such shifts to improve the efficacy of prediction models. By integrating Explainable AI (XAI) techniques into adaptive learning with SHAP clustering, we generate interpretable model explanations and use these insights for adaptive refinement. Our three-stage process: (1) SHAP value generation for the model explanation, (2) clustering these values for pattern identification, and (3) model refinement based on the derived SHAP cluster characteristics, mitigates overfitting and ensures robust data shift handling. We evaluate our method on a comprehensive dataset comprising energy consumption records of buildings, as well as two additional datasets, to assess the transferability of our approach to other domains, regression, and classification problems. Our experiments highlight that our method not only improves predictive performance in both task types but also delivers interpretable model explanations, offering significant value in dealing with the challenges of data distribution shifts in AI.

Authors with CRIS profile

Tobias Clement Lehrstuhl für Wirtschaftsinformatik, insbesondere IT-Management Hung Nguyen Truong Thanh Lehrstuhl für Wirtschaftsinformatik, insbesondere IT-Management Nils Kemmerzell Lehrstuhl für Wirtschaftsinformatik, insbesondere IT-Management

Related research project(s)

Towards a Better Understanding of Explainable Artificial Intelligence (XAI): Insights Gained from Selected Studies (XAI) Software Campus 2.0 April 1, 2022 - March 31, 2024

Involved external institutions

Software AG DE Germany (DE) Granlund FI Finland (FI)

How to cite

APA:

Clement, T., Nguyen Truong Thanh, H., Kemmerzell, N., Abdelaal, M., & Stjelja, D. (2024). Coping with Data Distribution Shifts: XAI-Based Adaptive Learning with SHAP Clustering for Energy Consumption Prediction. In Tongliang Liu, Geoff Webb, Lin Yue, Dadong Wang (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 147-159). Brisbane, QLD, AU: Springer Science and Business Media Deutschland GmbH.

MLA:

Clement, Tobias, et al. "Coping with Data Distribution Shifts: XAI-Based Adaptive Learning with SHAP Clustering for Energy Consumption Prediction." Proceedings of the 36th Australasian Joint Conference on Artificial Intelligence, AJCAI 2023, Brisbane, QLD Ed. Tongliang Liu, Geoff Webb, Lin Yue, Dadong Wang, Springer Science and Business Media Deutschland GmbH, 2024. 147-159.

BibTeX: Download