Confidence Estimation and Boosting for Dynamic-Comparator Transient-Noise Analysis - Conrad, Joschua; Kauffman, John; Wilhelmstatter, Simon; Asthana, Rohan; Belagiannis, Vasileios; Ortmanns, Maurits (2024 22nd IEEE Interregional NEWCAS Conference (NEWCAS), 2024)
While designing a comparator, simulation of input-referred noise is crucial. While periodic steady state analyses allow a fast simulation, they are cumbersome to set-up, not compatible with all kinds of circuit models, and require to know the noise-critical operating time within a period. On the contrary, the comparator can be analyzed in a noisy transient simulation with a small test-voltage applied. The noise and offset characteristics can then be derived from a number of simulated comparator decisions and a statistical model. This method is easy to set-up, but has two fundamental simulation parameters: the number of simulated comparator decisions and the magnitude of the applied test-voltage. In this work we derive the statistical confidence into the estimated input-referred noise and show how it depends on the two simulation parameters. It is then shown that this confidence also depends on the true (unknown) input-referred noise. To solve that issue, an algorithm for boosting the convergence is introduced. This algorithm iteratively simulates the comparator decisions and updates the test-voltage for granting a known confidence into the estimated input-referred noise. The algorithm implemented in Verilog-A is available under https://github.com/Joschua-Conrad/Transient-Simulation-of-Dynamic-Comparator-Noise.
CRIS-Team Scopus Importer:2025-02-04
Keywords: Confidence; Dynamic Comparator; Simulation; Transient Noise Analysis

Differentiable Cost Model for Neural-Network Accelerator Regarding Memory Hierarchy - Conrad, Joschua; Wilhelmstatter, Simon; Asthana, Rohan; Belagiannis, Vasileios; Ortmanns, Maurits (IEEE Transactions on Circuits and Systems I-Regular Papers, 2024)
Dedicated neural-network inference-processors improve latency and power of the computing devices. They use custom memory hierarchies that take into account the flow of operators present in neural networks and convolutional layers. For efficient implementation, such network topologies can greatly benefit from hardware-cost optimization using automated network-architecture search. Thereby, cost functions predict the suitability of a network topology for a given type of inference hardware. A dnas that optimizes both weights and topology in a single training requires cost models to be differentiable in the dimensions of weight and activation matrices. soa differentiable cost models require time-consuming system-level measurements or simulation results, or do not encounter the hardware structure at all. This work presents a simple yet effective procedure for deriving a differentiable neural-network-accelerator cost-model that is suitable for any type of accelerator. It is based on hardware-independent parameterization and a novel differentiable divide-ceil function, as well as hardware-specific modeling. The resulting differentiable model can be reconfigured to the actual hardware size and memory structure to predict the inference energy for an exact network topology. The modeling and prediction are demonstrated for a soa SRAM-based inference-accelerator and for the Eyeriss accelerator, inferring different soa neural networks, resulting in excellent agreement with measured hardware.
CRIS-Team Scopus Importer:2024-11-08
Keywords: differentiable cost model; differentiable network-architecture search; in-memory computing; network/hardware co-design; Neural network

Multi-conditioned Graph Diffusion for Neural Architecture Search - Asthana, Rohan; Conrad, Joschua; Dawoud, Youssef; Ortmanns, Maurits; Belagiannis, Vasileios (Transactions on Machine Learning Research, 2024)
Neural architecture search automates the design of neural network architectures usually by exploring a large and thus complex architecture search space. To advance the architecture search, we present a graph diffusion-based NAS approach that uses discrete conditional graph diffusion processes to generate high-performing neural network architectures. We then propose a multi-conditioned classifier-free guidance approach applied to graph diffusion networks to jointly impose constraints such as high accuracy and low hardware latency. Unlike the related work, our method is completely differentiable and requires only a single model training. In our evaluations, we show promising results on six standard benchmarks, yielding novel and unique architectures at a fast speed, i.e. less than 0.2 seconds per architecture. Furthermore, we demonstrate the generalisability and efficiency of our method through experiments on ImageNet dataset.<br
Keywords: deep learning; neural architecture search; diffusion models

Too-Hot-to-Handle: Insights into Temperature and Noise Hyperparameters for Differentiable Neural-Architecture-Searches - Conrad, Joschua; Wilhelmstatter, Simon; Asthana, Rohan; Belagiannis, Vasileios; Ortmanns, Maurits (2024 IEEE 6th International Conference on AI Circuits and Systems (AICAS), 2024)
The deployment of neural networks on resource-constrained hardware requires architecture optimization algorithms that respect accuracy and computational cost. One emerging way for performing such an architecture optimization is the differentiable neural-architecture-search (DNAS). DNAS finds the weights of a neural network as well as its topology in a single training. The topology search-space is represented by instantiating different layer-candidates for each layer in parallel, thereby forming a so-called weighted super-network. The instances are connected via a gumbel-softmax function which employs important parameters commonly called temperature τ and noise-scale β. The temperature is scheduled during training and controls the convergence to selecting only a single final candidate per optimized layer. The noise-scale adds a random behavior in trying different candidates before deciding for a final one. The state-of-the-art (SotA) does not define rules for setting those important hyperparameters. Therefore, this work elaborates methods to predict feasible start- and end-temperature as well as noise-scale prior to training and to fine-tune the parameters from metrics obtained during a DNAS.
CRIS-Team Scopus Importer:2024-08-09
Keywords: differentiable neural-architecture-search; gumbel-softmax; temperature