Resilience and Precision Assessment of Natural Language Processing Algorithms in Analog In-Memory Computing: A Hardware-Aware Study - Parvaresh, Amirhossein; Hosseinzadeh, Shima; Fey, Dietmar (ACM International Conference Proceeding Series, 2023)
Natural Language Processing (NLP) serves as a cornerstone technology, facilitating complex human-computer interactions, enabling information retrieval, conducting sentiment analysis, and enhancing language comprehension. With the ever-growing use of NLPs, the conventional 'von Neumann' computing paradigm is rapidly approaching its inherent limitations. In response, Analog In-Memory Computing (AIMC) emerges as a compelling alternative, albeit accompanied by inherent non-idealities when deploying neural networks on such platforms. In this paper, we have evaluated the precision and resilience of various NLP algorithms when executed within the AIMC framework, both with and without the application of hardware-aware training. Our analysis reveals noteworthy insights: Gated Recurrent Unit (GRU) neural networks exhibit enhanced resilience to noise, yielding an average test error of 3.97% following hardware-aware training, as compared to their full precision counterparts. Conversely, Long Short-Term Memory (LSTM) networks demonstrate a slightly higher average test error of 5.67%, indicating a relatively lower tolerance to non-idealities. In contrast, Convolutional Neural Networks (CNNs) manifest a heightened vulnerability, exhibiting an average relative test error of 13.34%. Furthermore, we systematically investigate the sensitivity profiles of the selected neural networks in the presence of specific non-idealities, providing valuable insights into their robustness and susceptibility within the AIMC environment.
CRIS-Team Scopus Importer:2024-12-16
Keywords: AIMC; DNN; NLP; PCM