Radio Foundation Models: Pre-training Transformers for 5G-based Indoor Localization

Ott J, Pirkl J, Stahlke M, Feigl T, Mutschler C (2024)

Publication Language: English

Publication Type: Conference contribution, Original article

Publication year: 2024

Pages Range: 1-6

Conference Proceedings Title: The fourteenth edition of the International Conference on Indoor Positioning and Indoor Navigation (IPIN)

Event location: Hong Kong, China

Abstract

Artificial Intelligence (AI)-based radio fingerprint-

ing (FP) outperforms classic localization methods in propagation

environments with strong multipath effects. However, the model

and data orchestration of FP are time-consuming and costly, as

it requires many reference positions and extensive measurement

campaigns for each environment. Instead, modern unsupervised

and self-supervised learning schemes require less reference data

for localization, but either their accuracy is low or they require

additional sensor information, rendering them impractical.

In this paper we propose a self-supervised learning framework

that pre-trains a general transformer (TF) neural network on

5G channel measurements that we collect on-the-fly without

expensive equipment. Our novel pretext task randomly masks

and drops input information to learn to reconstruct it. So, it

implicitly learns the spatiotemporal patterns and information of

the propagation environment that enable FP-based localization.

Most interestingly, when we optimize this pre-trained model for

localization in a given environment, it achieves the accuracy of

state-of-the-art methods but requires ten times less reference data

and significantly reduces the time from training to operation.

Authors with CRIS profile

Maximilian Stahlke Machine Learning and Data Analytics Lab Tobias Feigl Lehrstuhl für Informatik 2 (Programmiersysteme) Christopher Mutschler

Involved external institutions

Fraunhofer-Institut für Integrierte Schaltungen (IIS) DE Germany (DE)

How to cite

APA:

Ott, J., Pirkl, J., Stahlke, M., Feigl, T., & Mutschler, C. (2024). Radio Foundation Models: Pre-training Transformers for 5G-based Indoor Localization. In The fourteenth edition of the International Conference on Indoor Positioning and Indoor Navigation (IPIN) (pp. 1-6). Hong Kong, China.

MLA:

Ott, Jonathan, et al. "Radio Foundation Models: Pre-training Transformers for 5G-based Indoor Localization." Proceedings of the 2024 14th International Conference on Indoor Positioning and Indoor Navigation (IPIN), Hong Kong, China 2024. 1-6.

BibTeX: Download