Differentiable Cost Model for Neural-Network Accelerator Regarding Memory Hierarchy

Conrad J, Wilhelmstatter S, Asthana R, Belagiannis V, Ortmanns M (2024)

Publication Type: Journal article

Publication year: 2024

Journal

IEEE Transactions on Circuits and Systems I-Regular Papers Institute of Electrical and Electronics Engineers (IEEE)

DOI: 10.1109/TCSI.2024.3476534

Abstract

Dedicated neural-network inference-processors improve latency and power of the computing devices. They use custom memory hierarchies that take into account the flow of operators present in neural networks and convolutional layers. For efficient implementation, such network topologies can greatly benefit from hardware-cost optimization using automated network-architecture search. Thereby, cost functions predict the suitability of a network topology for a given type of inference hardware. A dnas that optimizes both weights and topology in a single training requires cost models to be differentiable in the dimensions of weight and activation matrices. soa differentiable cost models require time-consuming system-level measurements or simulation results, or do not encounter the hardware structure at all. This work presents a simple yet effective procedure for deriving a differentiable neural-network-accelerator cost-model that is suitable for any type of accelerator. It is based on hardware-independent parameterization and a novel differentiable divide-ceil function, as well as hardware-specific modeling. The resulting differentiable model can be reconfigured to the actual hardware size and memory structure to predict the inference energy for an exact network topology. The modeling and prediction are demonstrated for a soa SRAM-based inference-accelerator and for the Eyeriss accelerator, inferring different soa neural networks, resulting in excellent agreement with measured hardware.

Authors with CRIS profile

Rohan Asthana Professur für Maschinelles Lernen in der Signalverarbeitung Vasileios Belagiannis Professur für Maschinelles Lernen in der Signalverarbeitung

Involved external institutions

Universität Ulm DE Germany (DE)

How to cite

APA:

Conrad, J., Wilhelmstatter, S., Asthana, R., Belagiannis, V., & Ortmanns, M. (2024). Differentiable Cost Model for Neural-Network Accelerator Regarding Memory Hierarchy. IEEE Transactions on Circuits and Systems I-Regular Papers. https://doi.org/10.1109/TCSI.2024.3476534

MLA:

Conrad, Joschua, et al. "Differentiable Cost Model for Neural-Network Accelerator Regarding Memory Hierarchy." IEEE Transactions on Circuits and Systems I-Regular Papers (2024).

BibTeX: Download