LangXAI: Integrating Large Vision Models for Generating Textual Explanations to Enhance Explainability in Visual Perception Tasks

Nguyen H, Clement T, Nguyen L, Kemmerzell N, Truong B, Nguyen K, Abdelaal M, Cao H (2024)

Publication Language: English

Publication Type: Conference contribution

Publication year: 2024

Journal

IJCAI-97 - PROCEEDINGS OF THE FIFTEENTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOLS 1 AND 2

Publisher: International Joint Conferences on Artificial Intelligence

Pages Range: 8754-8758

Conference Proceedings Title: Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence

Event location: Jeju KR

ISBN: 9781956792041

Abstract

LangXAI is a framework that integrates Explainable Artificial Intelligence (XAI) with advanced vision models to generate textual explanations for visual recognition tasks. Despite XAI advancements, an understanding gap persists for end-users with limited domain knowledge in artificial intelligence and computer vision. LangXAI addresses this by furnishing text-based explanations for classification, object detection, and semantic segmentation model outputs to end-users. Preliminary results demonstrate LangXAI's enhanced plausibility, with high BERTScore across tasks, fostering a more transparent and reliable AI framework on vision tasks for end-users. The code and demo of this work can be found at https://analytics-everywhere-lab.github.io/langxai.io/.

Authors with CRIS profile

Tobias Clement Lehrstuhl für Wirtschaftsinformatik, insbesondere IT-Management Nils Kemmerzell Lehrstuhl für Wirtschaftsinformatik, insbesondere IT-Management

Involved external institutions

University of New Brunswick (UNB) CA Canada (CA) FPT Corporation VN Viet Nam (VN) Software AG DE Germany (DE)

How to cite

APA:

Nguyen, H., Clement, T., Nguyen, L., Kemmerzell, N., Truong, B., Nguyen, K.,... Cao, H. (2024). LangXAI: Integrating Large Vision Models for Generating Textual Explanations to Enhance Explainability in Visual Perception Tasks. In Kate Larson (Eds.), Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (pp. 8754-8758). Jeju, KR: International Joint Conferences on Artificial Intelligence.

MLA:

Nguyen, Hung, et al. "LangXAI: Integrating Large Vision Models for Generating Textual Explanations to Enhance Explainability in Visual Perception Tasks." Proceedings of the 33rd International Joint Conference on Artificial Intelligence, IJCAI 2024, Jeju Ed. Kate Larson, International Joint Conferences on Artificial Intelligence, 2024. 8754-8758.

BibTeX: Download