A Safari through FPGA-based Neural Network Compilation and Design Automation Flows - Plagwitz, Patrick; Hannig, Frank; Ströbel, Martin; Strohmeyer, Christoph; Teich, Jürgen (Proceedings of the 29th IEEE International Symposium on Field-Programmable Custom Computing Machines (FCCM), 2021)

Compiler-Based High-Level Synthesis of Application-Specific Processors on FPGAs - Plagwitz, Patrick; Streit, Franz-Josef; Becher, Andreas; Wildermann, Stefan; Teich, Jürgen (IEEE Proceedings of the 14th International Conference on ReConFigurable Computing and FPGAs, 2019)
In order to meet tight performance and/or energy constraints of embedded systems, the implementation of applications in hardware is often a must. However, mapping of algorithms to platforms, as for example Field-Programmable Gate Arrays (FPGAs), still requires comprehensive hardware knowledge and sometimes long design cycles. Modern High-Level Synthesis (HLS) offers a means to ease the generation of hardware implementations from a software specification of an application. Although these tools have improved greatly in recent years, they often do not provide full coverage of important programming constructs and are therefore of limited use when used with existing or automatically generated code. Soft-core processors implemented with FPGA-logic can circumvent this limitation. However, these come with drawbacks in terms of performance and resource requirements as a general-purpose architecture is used to implement the application in software rather than as a highly specialized circuit. As a remedy, our work presents a novel compiler-based synthesis methodology that generates networks of Application-Specific Instruction Set Processors (ASIPs) from unmodified C/C++ algorithms. We thereby bridge the gap between traditional soft-core processors and HLS. To show the practicability of our approach, we present a case study of a JPEG decoder application while investigating design objectives like resource costs and performance. Apart from the generality of the compiler-based approach, our approach also shows better results in terms of required hardware resources and execution times compared to Instruction Set Architecture (ISA)-fixed commercial Xilinx MicroBlaze soft-core
Keywords: FPGA; HLS; ASIP; SoC; Hardware/Software Co-Design

Compiler-based Processor Network Generation for Neural Networks on FPGAs - Plagwitz, Patrick; Hannig, Frank; Teich, Jürgen; Keszöcze, Oliver (Proceedings of the 27th Workshop on Methods and Description Languages for Modelling and Verification of Circuits and Systems (MBMV), 2024)
The fast-moving field of Neural Network (NN) research has brought breakthroughs in application domains like image or natural language processing. However, since new techniques are constantly emerging, fast prototyping of Neural Networks on edge devices like Field-Programmable Gate Arrays (FPGAs) is particularly appealing due to their reconfigurability. Existing compiler-based toolchains generating specialized hardware are rare and restricted in their scope and coverage. In this paper, we propose a novel NN model-to-hardware flow that exploits the capabilities of Application-Specific Instruction Set Processors (ASIPs) and networks that can be built to implement NNs in a scalable and mixed hardware/software-reconfigurable way. Starting with an NN specification in PyTorch, an ASIP network generation toolchain is presented in which each instantiated core is assigned a portion of neurons to execute a given NN in a parallel and pipelined way while tailoring the instruction set of each core to the computations of the neurons. In addition, we propose multiple ML-specific instruction set extensions realized in hardware, including bit width-restricted ALU operations and activation functions with the circuits automatically generated from a given C/C++ implementation. Experimental results include a design space exploration of alternative implementations for the design objectives of FPGA resource usage, latency, and throughput. Our approach is shown to achieve higher speed-ups than similar processor-based approaches while consuming lower amounts of resources.<br

DSL-based SNN Accelerator Design using Chisel - Plagwitz, Patrick; Hannig, Frank; Teich, Jürgen; Keszöcze, Oliver (Proceedings of the 27th Euromicro Conference on Digital Systems Design (DSD), 2024)
Spiking Neural Networks (SNNs) are a promising class of algorithms for hardware acceleration, even outperforming traditional neural networks in some cases. However, existing SNN accelerator approaches do not perform exhaustive explorations of possible network parameters, including neuron models and spike codings; instead, they often focus on a single network setting and a given fixed hardware architecture for its implementation. Chisel is a hardware construction language that allows the modeling of high-level abstractions from the Register-Transfer Level (RTL) and above. It promises a more transparent and predictable approach than compiler-based methodologies, like High-Level Synthesis (HLS). In this paper, we propose a novel multi-layer Domain-Specific Language (DSL) for SNN accelerator design based on Chisel, allowing for design space explorations that vary neuron models, spike codings, reset behaviors, and even accelerator topologies. Moreover, we propose an SNN accelerator generation framework using this DSL, which covers training to deployment. We explore and evaluate implementations and provide results regarding execution time, Field-Programmable Gate Array (FPGA) resource usage, power consumption, and accuracy.
Keywords: FPGAs; DSL; Machine Learning; Spiking Neural Networks

SNN vs. CNN Implementations on FPGAs: An Empirical Evaluation - Plagwitz, Patrick; Hannig, Frank; Teich, Jürgen; Keszöcze, Oliver (Proceedings of the 20th International Symposium on Applied Reconfigurable Computing. Architectures, Tools, and Applications (ARC), 2024)
Convolutional Neural Networks (CNNs) are widely employed to solve various problems, e.g., image classification. Due to their compute- and data-intensive nature, CNN accelerators have been developed as ASICs or on FPGAs. The increasing complexity of applications has caused resource costs and energy requirements of these accelerators to grow. Spiking Neural Networks (SNNs) are an emerging alternative to CNN implementations, promising higher resource and energy efficiency. The main research question addressed in this paper is whether SNN accelerators truly meet these expectations of reduced energy demands compared to their CNN equivalents when implemented on modern FPGAs. For this purpose, we analyze multiple SNN hardware accelerators for FPGAs regarding performance and energy efficiency. We also present a novel encoding scheme of spike event queues and a novel memory organization technique to improve SNN energy efficiency further. Both techniques have been integrated into a state-of-the-art SNN architecture and evaluated for MNIST, SVHN, and CIFAR-10 data sets and corresponding network architectures on two differently sized modern FPGA platforms. A result of our empirical analysis is that for complex benchmarks such as SVHN and CIFAR-10, SNNs do live up to their expectations.<br

To Spike or Not to Spike? A Quantitative Comparison of SNN and CNN FPGA Implementations - Plagwitz, Patrick; Hannig, Frank; Teich, Jürgen; Keszöcze, Oliver (2023)
Convolutional Neural Networks (CNNs) are widely employed to solve various problems, e.g., image classification. Due to their compute- and data-intensive nature, CNN accelerators have been developed as ASICs or on FPGAs. Increasing complexity of applications has caused resource costs and energy requirements of these accelerators to grow. Spiking Neural Networks (SNNs) are an emerging alternative to CNN implementations, promising higher resource and energy efficiency. The main research question addressed in this paper is whether SNN accelerators truly meet these expectations of reduced energy requirements compared to their CNN equivalents. For this purpose, we analyze multiple SNN hardware accelerators for FPGAs regarding performance and energy efficiency. We present a novel encoding scheme of spike event queues and a novel memory organization technique to improve SNN energy efficiency further. Both techniques have been integrated into a state-of-the-art SNN architecture and evaluated for MNIST, SVHN, and CIFAR-10 datasets and corresponding network architectures on two differently sized modern FPGA platforms. For small-scale benchmarks such as MNIST, SNN designs provide rather no or little latency and energy efficiency advantages over corresponding CNN implementations. For more complex benchmarks such as SVHN and CIFAR-10, the trend reverses

TRAC: Compilation-based Design of Transformer Accelerators for FPGAs - Plagwitz, Patrick; Hannig, Frank; Teich, Jürgen (IEEE Proceedings of the 32nd International Conference on Field Programmable Logic and Applications, 2022)