RT-SCNNs: Real-time spiking convolutional neural networks for a novel hand gesture recognition using time-domain mm-wave radar data

Shaaban A, Strobel M, Furtner W, Weigel R, Lurz F (2024)

Publication Type: Journal article

Publication year: 2024

Journal

International Journal of Microwave and Wireless Technologies Cambridge University Press

Book Volume: 16

Pages Range: 783-795

Journal Issue: 5

DOI: 10.1017/S1759078723001575

Abstract

This study introduces a novel approach to radar-based hand gesture recognition (HGR), addressing the challenges of energy efficiency and reliability by employing real-time gesture recognition at the frame level. Our solution bypasses the computationally expensive preprocessing steps, such as 2D fast Fourier transforms (FFTs), traditionally employed for range-Doppler information generation. Instead, we capitalize on time-domain radar data and harness the energy-efficient capabilities of spiking neural networks (SNNs) models, recognized for their sparsity and spikes-based communication, thus optimizing the overall energy efficiency of our proposed solution. Experimental results affirm the effectiveness of our approach, showcasing significant classification accuracy on the test dataset, with peak performance achieving a mean accuracy of 99.75%. To further validate the reliability of our solution, individuals who have not participated in the dataset collection conduct real-time live testing, demonstrating the consistency of our theoretical findings. Real-time inference reveals a substantial degree of spikes sparsity, ranging from 75% to 97%, depending on the presence or absence of a performed gesture. By eliminating the computational burden of preprocessing steps and leveraging the power of (SNNs), our solution presents a promising alternative that enhances the performance and usability of radar-based (HGR) systems.

Authors with CRIS profile

Ahmed Shaaban Lehrstuhl für Technische Elektronik (LTE) Robert Weigel Lehrstuhl für Technische Elektronik (LTE) Fabian Lurz Lehrstuhl für Technische Elektronik (LTE)

Involved external institutions

Infineon Technologies AG DE Germany (DE)

How to cite

APA:

Shaaban, A., Strobel, M., Furtner, W., Weigel, R., & Lurz, F. (2024). RT-SCNNs: Real-time spiking convolutional neural networks for a novel hand gesture recognition using time-domain mm-wave radar data. International Journal of Microwave and Wireless Technologies, 16(5), 783-795. https://doi.org/10.1017/S1759078723001575

MLA:

Shaaban, Ahmed, et al. "RT-SCNNs: Real-time spiking convolutional neural networks for a novel hand gesture recognition using time-domain mm-wave radar data." International Journal of Microwave and Wireless Technologies 16.5 (2024): 783-795.

BibTeX: Download