Gui Z, Freund M, Harth A (2024)
Publication Language: English
Publication Type: Conference contribution
Publication year: 2024
Pages Range: 4
Conference Proceedings Title: IoT '24: Proceedings of the 13th International Conference on the Internet of Things
Event location: Oulu, Finland
This paper presents a demonstration of a context-aware conversational interface for interacting with multiple IoT devices using multi-turn natural language commands. Our system facilitates device discovery and identification through a Knowledge Graph (KG) and addresses the interoperability issue by using the Web of Things (WoT) specifications as an intermediate abstraction layer. The proposed system comprises four major components: a context-aware multi-turn dialogue interface, a device discovery and identification module using a domain-specific KG, a text-to-code module to parse natural language commands into an executable code format, and a customized code execution engine. We demonstrate our system using two Philips Hue smart lamps and one Elgato Stream Deck controller. The Philips Hue smart lamp device can be controlled standalone via natural language, allowing users to adjust the power state, the color, and the brightness. Also, the lamp can be integrated with the Elgato Stream Deck Controller to enable user-defined automation rules following the Trigger Action programming (TAP) paradigm.
APA:
Gui, Z., Freund, M., & Harth, A. (2024). A Context-aware Conversational Interface for Controlling Internet-of-Things Devices. In IoT '24: Proceedings of the 13th International Conference on the Internet of Things (pp. 4). Oulu, Finland.
MLA:
Gui, Zhou, Michael Freund, and Andreas Harth. "A Context-aware Conversational Interface for Controlling Internet-of-Things Devices." Proceedings of the IoT 2024: 14th International Conference on the Internet of Things, Oulu, Finland 2024. 4.
BibTeX: Download