Towards an IMU-based Pen Online Handwriting Recognizer

Wehbi M, Hamann T, Barth J, Kaempf P, Zanca D, Eskofier B (2021)

Publication Type: Conference contribution, Conference Contribution

Publication year: 2021

Journal

Lecture Notes in Computer Science Springer Verlag

Original Authors: Mohamad Wehbi, Tim Hamann, Jens Barth, Peter Kaempf, Dario Zanca, Bjoern Eskofier

Publisher: Springer Link

Pages Range: 289-303

Event location: Lausanne CH

DOI: 10.1007/978-3-030-86334-0_19

Abstract

Most online handwriting recognition systems require the use of specific writing surfaces to extract positional data. In this paper we present a online handwriting recognition system for word recognition which is based on inertial measurement units (IMUs) for digitizing text written on paper. This is obtained by means of a sensor-equipped pen that provides acceleration, angular velocity, and magnetic forces streamed via Bluetooth. Our model combines convolutional and bidirectional LSTM networks, and is trained with the Connectionist Temporal Classification loss that allows the interpretation of raw sensor data into words without the need of sequence segmentation. We use a dataset of words collected using multiple sensor-enhanced pens and evaluate our model on distinct test sets of seen and unseen words achieving a character error rate of 17.97% and 17.08%, respectively, without the use of a dictionary or language model.

Authors with CRIS profile

Mohamad Wehbi Machine Learning and Data Analytics Lab Dario Zanca Machine Learning and Data Analytics Lab Björn Eskofier Machine Learning and Data Analytics Lab

Related research project(s)

Development of neural networks and machine learning algorithms for online handwriting recognition (EINNS) Development of neural networks and machine learning algorithms for online handwriting recognition Sept. 2, 2019 - April 30, 2022

Involved external institutions

STABILO International GmbH DE Germany (DE)

How to cite

APA:

Wehbi, M., Hamann, T., Barth, J., Kaempf, P., Zanca, D., & Eskofier, B. (2021). Towards an IMU-based Pen Online Handwriting Recognizer. In Proceedings of the International Conference on Document Analysis and Recognition ICDAR 2021 (pp. 289-303). Lausanne, CH: Springer Link.

MLA:

Wehbi, Mohamad, et al. "Towards an IMU-based Pen Online Handwriting Recognizer." Proceedings of the International Conference on Document Analysis and Recognition ICDAR 2021, Lausanne Springer Link, 2021. 289-303.

BibTeX: Download