Enhancing IMU-Based Online Handwriting Recognition via Contrastive Learning with Zero Inference Overhead
Li J, Zanca D, Christlein V, Hamann T, Barth J, Kämpf P, Eskofier B (2026)
Publication Type: Conference contribution
Publication year: 2026
Journal
Publisher: Springer Science and Business Media Deutschland GmbH
Book Volume: 16974 LNCS
Pages Range: 695-711
Conference Proceedings Title: Lecture Notes in Computer Science
Event location: Vienna, AUT
ISBN: 9783032360380
DOI: 10.1007/978-3-032-36039-7_41
Abstract
Online handwriting recognition from inertial measurement units enables handwriting on paper as input for digital devices. Running recognition on edge hardware improves privacy and lowers latency, but entails memory constraints. To address this, we propose Error-Aware Contrastive-Enhanced Handwriting Recognition (ECHWR), a training-time contrastive learning framework designed to improve recognition accuracy without increasing inference costs. During training, ECHWR utilizes a temporary auxiliary branch that aligns IMU signal embeddings with semantic text embeddings in a shared space. This is enforced with a dual objective: (i) an in-batch contrastive loss that aligns IMU and text embeddings, and (ii) a hard-negative contrastive loss that uses synthetically perturbed text targets. The auxiliary branch is discarded after training, which allows the deployed model to keep its original, efficient architecture. Experiments on the OnHW-Words500 dataset show that ECHWR significantly outperforms state-of-the-art baselines, reducing character error rates by up to 7.4% on the writer-independent split and 10.4% on the writer-dependent split. Ablations indicate that the hard-negative objective is particularly effective for generalization to unseen writing styles, making our approach well-suited for online handwriting recognition on unseen users. Code is available at: https://github.com/jindongli24/ECHWR.
Involved external institutions
Germany (DE)How to cite
APA:
Li, J., Zanca, D., Christlein, V., Hamann, T., Barth, J., Kämpf, P., & Eskofier, B. (2026). Enhancing IMU-Based Online Handwriting Recognition via Contrastive Learning with Zero Inference Overhead. In Gernot A. Fink, Alicia Fornés, Koichi Kise, Daniel Lopresti (Eds.), Lecture Notes in Computer Science (pp. 695-711). Vienna, AUT: Springer Science and Business Media Deutschland GmbH.
MLA:
Li, Jindong, et al. "Enhancing IMU-Based Online Handwriting Recognition via Contrastive Learning with Zero Inference Overhead." Proceedings of the 20th International Conference on Document Analysis and Recognition, ICDAR 2026, Vienna, AUT Ed. Gernot A. Fink, Alicia Fornés, Koichi Kise, Daniel Lopresti, Springer Science and Business Media Deutschland GmbH, 2026. 695-711.
BibTeX: Download