Contributions to Visual Automotive Human Machine Interface Testing

Pöllot M (2020)

Publication Language: English

Publication Type: Thesis

Publication year: 2020

Publisher: Dr. Hut

City/Town: München

ISBN: 978-3-8439-4647-6



The amount of driver assistance, entertainment, and information features in automobiles is increasing rapidly over the past few years. Human machine interfaces are the means to communicate between car and passengers. With the regard of automated testing of these interfaces, a typical scenario in the development of automotive infotainment systems is revealed. Conventional approaches for testing, based on empirical thresholds, however, are not able to provide reliable detection rates and require a lot of maintenance. This is where model-based testing approaches come into play. By applying model-based approaches, the acquisition of empirical thresholds is abandoned in favor of models learned by algorithms. Relying on an underlying model, the algorithms exhibit higher capabilities in detecting errors that approaches based on empirical thresholds are not taking into account.
The advent of powerful machine learning approaches plays an important role in the data-based detection of errors in infotainment systems. Conventional approaches require prior knowledge of errors in order to develop custom-made algorithms for their detections. As many errors pass the development stage without being noticed due to never being observed during this time, a novel approach for automated testing human machine interfaces is advised.
This thesis concentrates on automated testing in the context of a model-based end-to-end approach. After introducing the main approaches and components for evaluating and setting up an automated testing environment, a first component for interacting with the infotainment system is provided, which could bring human-like input patterns forward. The main part of this thesis focuses on the detection of irregular motion in the navigation context. By applying a model-based approach that learns the normal behavior of the system, irregularities could be found with relative ease in comparison to stale and fixed threshold-based approaches. Furthermore, in the domain of screen content verification, a deep neural network approach is dedicated to detect and classify icons in a given screen. This approach could speed up the state-of-the-art sliding window pattern matching approach and also be more robust to design changes and noise. All contributions provide extensive experiments on custom data sets and demonstrate the advantage over conventional methods.
Three model-based contributions and two prototypes for screen content testing are investigated and adapted to provide improved results over threshold-based approaches. The applications comprise screen input testing, screen content analysis and screen content verification. All methods are superior to their respective reference methods in terms of detection rates, execution time and required maintenance, thus providing benefit in automated test environments.

Authors with CRIS profile

How to cite


Pöllot, M. (2020). Contributions to Visual Automotive Human Machine Interface Testing (Dissertation).


Pöllot, Martin. Contributions to Visual Automotive Human Machine Interface Testing. Dissertation, München: Dr. Hut, 2020.

BibTeX: Download