Memristor-only LSTM Acceleration with Non-linear Activation Functions
de Moura RF, de Lima JPC, Carro L (2023)
Publication Type: Conference contribution
Publication year: 2023
Journal
Publisher: Springer Science and Business Media Deutschland GmbH
Book Volume: 669
Pages Range: 89-100
Conference Proceedings Title: IFIP Advances in Information and Communication Technology
Event location: Lippstadt, DEU
ISBN: 9783031342134
DOI: 10.1007/978-3-031-34214-1_8
Abstract
Long Short-Term Memories (LSTMs) applied to Speech Recognition are an essential application of modern embedded devices. Computing Matrix-Vector Multiplications (MVMs) with Resistive Random Access Memory (ReRAM) crossbars has paved the way for solving the memory bottleneck issues related to LSTM processing. However, mixed-signal and fully-analog accelerators still lack in developing energy-efficient and versatile devices for the calculus of activation functions between MVM operations. This paper proposes a design methodology and circuitry that achieves both energy efficiency and versatility by introducing a programmable memristor array for computing nonlinearities. We exploit the inherent capability of ReRAM crossbars in computing MVM to perform piecewise linear interpolation (PWL) of non-linear activation functions, achieving a programmable device with a smaller cost. Experiments show that our approach outperforms state-of-the-art LSTM accelerators being 4.85x more efficient using representative speech recognition datasets.
Involved external institutions
Brazil (BR)How to cite
APA:
de Moura, R.F., de Lima, J.P.C., & Carro, L. (2023). Memristor-only LSTM Acceleration with Non-linear Activation Functions. In Stefan Henkler, Achim Rettberg, Márcio Kreutz, Marco A. Wehrmeister, Marcelo Götz (Eds.), IFIP Advances in Information and Communication Technology (pp. 89-100). Lippstadt, DEU: Springer Science and Business Media Deutschland GmbH.
MLA:
de Moura, Rafael Fão, João Paulo C. de Lima, and Luigi Carro. "Memristor-only LSTM Acceleration with Non-linear Activation Functions." Proceedings of the 7th IFIP TC 10 International Embedded Systems Symposium, IESS 2022, Lippstadt, DEU Ed. Stefan Henkler, Achim Rettberg, Márcio Kreutz, Marco A. Wehrmeister, Marcelo Götz, Springer Science and Business Media Deutschland GmbH, 2023. 89-100.
BibTeX: Download