Resilience and Precision Assessment of Natural Language Processing Algorithms in Analog In-Memory Computing: A Hardware-Aware Study
Parvaresh A, Hosseinzadeh S, Fey D (2023)
Publication Type: Conference contribution
Publication year: 2023
Publisher: Association for Computing Machinery
Conference Proceedings Title: ACM International Conference Proceeding Series
Event location: Dresden, DEU
ISBN: 9798400703256
Abstract
Natural Language Processing (NLP) serves as a cornerstone technology, facilitating complex human-computer interactions, enabling information retrieval, conducting sentiment analysis, and enhancing language comprehension. With the ever-growing use of NLPs, the conventional 'von Neumann' computing paradigm is rapidly approaching its inherent limitations. In response, Analog In-Memory Computing (AIMC) emerges as a compelling alternative, albeit accompanied by inherent non-idealities when deploying neural networks on such platforms. In this paper, we have evaluated the precision and resilience of various NLP algorithms when executed within the AIMC framework, both with and without the application of hardware-aware training. Our analysis reveals noteworthy insights: Gated Recurrent Unit (GRU) neural networks exhibit enhanced resilience to noise, yielding an average test error of 3.97% following hardware-aware training, as compared to their full precision counterparts. Conversely, Long Short-Term Memory (LSTM) networks demonstrate a slightly higher average test error of 5.67%, indicating a relatively lower tolerance to non-idealities. In contrast, Convolutional Neural Networks (CNNs) manifest a heightened vulnerability, exhibiting an average relative test error of 13.34%. Furthermore, we systematically investigate the sensitivity profiles of the selected neural networks in the presence of specific non-idealities, providing valuable insights into their robustness and susceptibility within the AIMC environment.
Authors with CRIS profile
Shima Hosseinzadeh Foroushani
Lehrstuhl für Informatik 3 (Rechnerarchitektur)
Dietmar Fey
Lehrstuhl für Informatik 3 (Rechnerarchitektur)
Involved external institutions
Germany (DE)How to cite
APA:
Parvaresh, A., Hosseinzadeh, S., & Fey, D. (2023). Resilience and Precision Assessment of Natural Language Processing Algorithms in Analog In-Memory Computing: A Hardware-Aware Study. In ACM International Conference Proceeding Series. Dresden, DEU: Association for Computing Machinery.
MLA:
Parvaresh, Amirhossein, Shima Hosseinzadeh, and Dietmar Fey. "Resilience and Precision Assessment of Natural Language Processing Algorithms in Analog In-Memory Computing: A Hardware-Aware Study." Proceedings of the 18th ACM International Symposium on Nanoscale Architectures, NANOARCH 2023, Dresden, DEU Association for Computing Machinery, 2023.
BibTeX: Download