A Deeply Pipelined and Parallel Architecture for Denoising Medical Images - Hannig, Frank; Schmid, Moritz; Teich, Jürgen; Hornegger, Heinz (Proc. IEEE International Conference on Field Programmable Technology, 2010)
In this paper we present an almost automatic synthesis of a highly complex, throughput optimized architecture of an adaptive multiresolution filter as used in medical image processing for FPGAs. The filter consists of 16 parallel working modules, where the most computationally intensive module achieves software pipelining of a factor of 85, that is, computations of 85 iterations overlap each other. By applying a state-of-the-art high-level synthesis tool, we show that this approach can be used for real world applications. In addition, we show that our high-level synthesis tool is capable of significantly reducing the well known productivity gap of embedded system design by almost two orders of magnitude. Finally, we can conclude that the FPGA implementation of the multiresolution image processing algorithm is far ahead of a comparable implementation for graphics cards in terms of power efficiency. © 2010 IEEE.
UnivIS-Import:2015-04-16:Pub.2010.tech.IMMD.inform.adeepl

An Image Processing Library for C-based High-Level Synthesis - Schmid, Moritz; Apelt, Nicolas; Hannig, Frank; Teich, Jürgen (Proc. of Field-Programmable Logic and Applications (FPL), 2014)
We introduce a library for the productive development of image processing accelerators using C-based high-level synthesis. The key concept of our approach is to provide a set of generic building blocks that is applicable to a multitude of image processing applications. An efficient memory architecture that facilitates easy integration of point and local image processing operators is the centerpiece of the library. The generic building blocks are kept very compact and can be tailored to support sophisticated processing techniques. The representation enables the designer to comply with specific design requirements, such as stringent timing constraints or limited resource budgets. Results show a significant gain in productivity compared to hand coded implementation while delivering comparable performance and resource requirements.
UnivIS-Import:2015-04-16:Pub.2014.tech.IMMD.inform.animag

A Novel Image Impulse Noise Removal Algorithm Optimized for Hardware Accelerators - Bhadouria, Vivek Singh; Tanase, Alexandru-Petru; Schmid, Moritz; Hannig, Frank; Teich, Jürgen; Ghoshal, Dibyendu (Journal of Signal Processing Systems, 2016)

Big Data and HPC Acceleration with Vivado HLS - Schmid, Moritz; Schmitt, Christian; Hannig, Frank; Malazgirt, Gorker Alp; Sönmez, Nehir; Yurdakul, Arda; Cristal, Adrián (FPGAs for Software Programmers, 2016)

Code Generation for High-Level Synthesis of Multiresolution Applications on FPGAs - Schmid, Moritz; Reiche, Oliver; Schmitt, Christian; Hannig, Frank; Teich, Jürgen (Proc. of the First International Workshop on FPGAs for Software Programmers (FSP), 2014)
Multiresolution Analysis (MRA) is a mathematical method that is based on working on a problem at different scales. One of its applications is medical imaging where processing at multiple scales—based on the concept of Gaussian and Laplacian image pyramids—is a well-known technique. It is often applied to reduce noise while preserving image detail on different levels of granularity without modifying the filter kernel. In scientific computing, multigrid methods are a popular choice, as they are asymptotically optimal solvers for elliptic Partial Differential Equations (PDEs). As such algorithms have a very high computational complexity that would overwhelm CPUs in the presence of real-time constraints, application-specific processors come into consideration for implementation. Despite of huge advancements in leveraging productivity in the respective fields, designers are still required to have detailed knowledge about coding techniques and the targeted architecture to achieve efficient solutions. Recently, the HIPAcc framework was proposed as a means for automatic code generation of image processing algorithms, based on a Domain-Specific Language (DSL). From the same code base, it is possible to generate code for efficient implementations on several accelerator technologies including different types of Graphics Processing Units (GPUs) as well as reconfigurable logic (FPGAs). In this work, we demonstrate the ability of HIPAcc to generate code for the implementation of multiresolution applications on FPGAs and embedded GPUs.
UnivIS-Import:2015-04-16:Pub.2014.tech.IMMD.inform.codege_1
Keywords: GRK-1773; ExaStencils

Code Generation from a Domain-specific Language for C-based HLS of Hardware Accelerators - Reiche, Oliver; Schmid, Moritz; Hannig, Frank; Membarth, Richard; Teich, Jürgen (Proceedings of the International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS), 2014)
As today's computer architectures are becoming more and more heterogeneous, a plethora of options including CPUs, GPUs, DSPs, reconfigurable logic (FPGAs), and other application-specific processors come into consideration for close-to-sensor processing. Especially, in the domain of image processing on mobile devices, among numerous design challenges, a very stringent energy budget is of utmost importance, making embedded GPUs and FPGAs ideal targets for implementation. Recently, the HIPAcc framework was proposed as a means for automatic code generation of image processing algorithms for embedded GPUs, based on a Domain-Specific Language (DSL). Despite of huge advancements in High-Level Synthesis (HLS) for FPGAs, designers are still required to have detailed knowledge about coding techniques and the targeted architecture to achieve efficient solutions. As a remedy, in this work, we propose code generation techniques for C-based HLS from a common high-level DSL description targeting FPGAs. Our approach includes FPGA-specific memory architectures for handling point and local operators, numerous high-level transformations, and automatic test bench generation. We evaluate our approach by comparing the resulting hardware accelerators to existing frameworks in terms of performance and resource requirements. Moreover, we assess the achieved energy efficiency in contrast to software implementations, generated by HIPAcc from the same code base, executed on GPUs. Copyright 2014 AC
UnivIS-Import:2015-04-16:Pub.2014.tech.IMMD.inform.codege
Keywords: Code generation; Domain-specific language; Image processing; High-level synthesis; Hardware accelerator; FPGA; GPU

Discourse on Extending Embedded Medical Image Processing Systems Using the High Speed Serial RapidIO Interconnect - Schmid, Moritz; Hannig, Frank; Teich, Jürgen; Diefenbach, Ralf; Pettendorf, Ralf; Hornegger, Heinz (Proceedings of the Embedded World Conference, 2010)

Domain-Specific Augmentations for High-Level Synthesis - Schmid, Moritz; Tanase, Alexandru-Petru; Hannig, Frank; Teich, Jürgen; Bhadouria, Vivek Singh; Ghoshal, Dibyendu (Proc. of the 25th IEEE International Conference on Application-specific Systems, Architectures and Processors (ASAP), 2014)
High-Level Synthesis (HLS) has become a very popular instrument to facilitate rapid development of production-ready implementations for FPGAs. Ever increasing flexibility of the frameworks, however, demands a very high level of domain-specific knowledge from the designer. Examples for such knowledge in window-based image processing are median computation and border handling. Depending on the size of the considered window, writing the code to perform such operations may become overwhelming even at very high abstraction levels. To increase productivity and to make the underlying architecture accessible to non-experts, we propose to combine HLS with domain-specific augmentations. Specifically, we propose a new language extension in form of a reduction for sorting and median computation. Furthermore, we introduce a new high-level transformation to perform multiple kinds of border treatment automatically. Both augmentations may reduce the required amount of code lines considerably. The increase in productivity is analyzed by comparing the lines of code necessary to specify a median filter for HLS in PAULA for synthesis using PARO and in C++ for synthesis using a commercial HLS tool. © 2014 IEEE.
UnivIS-Import:2015-04-16:Pub.2014.tech.IMMD.inform.domain

Generation of Multigrid-based Numerical Solvers for FPGA Accelerators - Schmitt, Christian; Schmid, Moritz; Hannig, Frank; Teich, Jürgen; Kuckuk, Sebastian; Köstler, Harald (Proceedings of the 2nd International Workshop on High-Performance Stencil Computations (HiStencils), 2015)
Not only in the field of high-performance computering, field-programmable gate arrays (FPGAs) are a soaringly popular accelerator technology. However, they increase the heterogeneity of clusters, which might be equipped already today with accelerators, such as GPUs. This results in having to combine expertise from different fields, e. g., mathematical, algorithmic and technical experts are needed to create numerical solvers for such systems. To bridge this programmability gap, domain-specific languages are a popular choice to generate low-level implementations from an abstract algorithm description. In this work, we demonstrate the generation of implementations of numerical solvers based on the multigrid method  for FPGAs from the same codebase that is also used to generate code for CPUs using a hybrid parallelization of MPI and OpenMP. Our approach yields in a hardware design that can compute up to 12 V-cycles per second with an input grid size of 4096x4096 on a mid-range FPGA, beating vectorized, single-threaded execution on an Intel i7 by a factor of almost three.

Hierarchical Power Management for Adaptive Tightly-Coupled Processor Arrays - Lari, Vahid; Muddasani, Shravan; Boppu, Srinivas; Hannig, Frank; Schmid, Moritz; Teich, Jürgen (ACM Transactions on Design Automation of Electronic Systems, 2012)
We present a self-adaptive hierarchical power management technique for massively parallel processor architectures, supporting a new resource-aware parallel computing paradigm called invasive computing. Here, an application can dynamically claim, execute, and release the resources in three phases: resource acquisition (invade), program loading/configuration and execution (infect), and release (retreat). Resource invasion is governed by dedicated decentralized hardware controllers, called invasion controllers (iCtrls), which are integrated into each processing element (PE). Several invasion strategies for claiming linearly connected or rectangular regions of processing resources are implemented. The key idea is to exploit the decentralized resource management inherent to invasive computing for power savings by enabling applications themselves to control the power for processing resources and invasion controllers using a hierarchical power-gating approach. We propose analytical models for estimating various components of energy consumption for faster design space exploration and compare them with the results obtained from a cycle-accurate C++ simulator of the processor array. In order to find optimal design trade-offs, various parameters like (a) energy consumption, (b) hardware cost, and (c) timing overheads are compared for different sizes of power domains. Experimental results show significant energy savings (up to 73%) for selected characteristical algorithms and different resource utilizations. In addition, we demonstrate the accuracy of our proposed analytical model. Here, estimation errors less than 3.6% can be reported. © 2012 ACM.
UnivIS-Import:2015-03-09:Pub.2012.tech.IMMD.inform.hierar
Keywords: Adaptive power optimization; Parallel computing; Resource awareness; Runtime resource management; Timing overhead minimization

High-Level Synthesis Revised - Generation of FPGA Accelerators from a Domain-Specific Language using the Polyhedron Model - Schmid, Moritz; Hannig, Frank; Tanase, Alexandru-Petru; Teich, Jürgen (Parallel Computing: Accelerating Computational Science and Engineering (CSE), 2014)
The continuous progress in semiconductor technology allows for more and more complex processor architectures. The downside of these technological advances is that computing has already hit a power wall and clock frequencies can barely be increased. In order to scale computing performance in the future, systems' energy efficiency and the degree of parallelism have to be significantly improved. The design of heterogeneous hardware with different specialized resources seems to be a promising solution. When highest performance (throughput, short latencies) and energy efficiency are important, as a remedy, we consider the generation of dedicated FPGA accelerators to address these stringent requirements. In this work, we present the PARO high-level synthesis framework for the automated generation of massively parallel FPGA accelerators. The framework is tailored for compute-intensive applications from the domains of image, video, and other digital signal processing, as well as algorithms from linear algebra. Unique features of PARO include: (1) The design entry in form of a compact and intuitive domain-specific language that is closely related to a mathematical problem description, (2) support for integer, fixed point, floating point, and custom arithmetic, (3) advanced loop transformations (e.g., partitioning) and scheduling techniques in the polyhedron model, (4)generation of accelerator IP cores (VHDL code) that can be easily integrated into a system design such as an SoC or in a networked scenario. Finally, we showcase the capabilities of our framework for the development of a range image conditioning pipeline for smart cameras for range sensing. © 2014 The authors and IOS Press.
UnivIS-Import:2015-04-20:Pub.2014.tech.IMMD.inform.highle
Keywords: Accelerators; Domain-specific language; FPGAs; High-level synthesis; Polyhedron model

HIPAcc - Schmid, Moritz; Reiche, Oliver; Hannig, Frank; Teich, Jürgen (FPGAs for Software Programmers, 2016)

Loop Coarsening in C-based High-Level Synthesis - Schmid, Moritz; Reiche, Oliver; Hannig, Frank; Teich, Jürgen (Proceedings of the 26th IEEE International Conference on Application-specific Systems, Architectures and Processors (ASAP), 2015)
Current tools for High-Level Synthesis (HLS) excel at exploiting Instruction-Level Parallelism (ILP), the support for Data-Level Parallelism (DLP), one of the key advantages of Field Programmable Gate Arrays (FPGAs), is in contrast very limited. This work examines the exploitation of DLP on FPGAs using code generation for C-based HLS of image filters and streaming pipelines, consisting of point and local operators. In addition to well known loop tiling techniques, we propose loop coarsening, which delivers superior performance and scalability. Loop tiling corresponds to splitting an image into separate regions, which are then processed in parallel by replicated accelerators. For data streaming, this also requires the generation of glue logic for the distribution of image data. Conversely, loop coarsening allows to process multiple pixels in parallel, whereby only the kernel operator is replicated within a single accelerator. We augment the FPGA back end of the heterogeneous Domain-Specific Language (DSL) framework HIPAcc by loop coarsening and compare the resulting FPGA accelerators to highly optimized software implementations for Graphics Processing Units (GPUs), all generated from the exact same code base. Moreover, we demonstrate the advantages of code generation for algorithm development by outlining how design space exploration enabled by HIPAcc can yield a more efficient implementation than handcoded VHDL.
UnivIS-Import:2015-11-17:Pub.2015.tech.IMMD.inform.loopco
Keywords: GRK-1773

Loop Parallelization Techniques for FPGA Accelerator Synthesis - Reiche, Oliver; Özkan, Mehmet Akif; Hannig, Frank; Teich, Jürgen; Schmid, Moritz (Journal of Signal Processing Systems, 2018)
Current tools for High-Level Synthesis (HLS) excel at exploiting Instruction-Level Parallelism (ILP). The support for Data-Level Parallelism (DLP), one of the key advantages of Field Programmable Gate Arrays (FPGAs), is in contrast very limited. This work examines the exploitation of DLP on FPGAs using code generation for C-based HLS of image filters and streaming pipelines. In addition to well-known loop tiling techniques, we propose loop coarsening, which delivers superior performance and scalability. Loop tiling corresponds to splitting an image into separate regions, which are then processed in parallel by replicated accelerators. For data streaming, this also requires the generation of glue logic for the distribution of image data. Conversely, loop coarsening allows processing multiple pixels in parallel, whereby only the kernel operator is replicated within a single accelerator. We present concrete implementations of tiling and coarsening for Vivado HLS and Altera OpenCL. Furthermore, we present a comparison of our implementations to the keyword-driven parallelization support provided by the Altera Offline Compiler. We augment the FPGA back end of the heterogeneous Domain-Specific Language (DSL) framework Hipacc to generate loop coarsening implementations for Vivado HLS and Altera OpenCL. Moreover, we compare the resulting FPGA accelerators to highly optimized software implementations for Graphics Processing Units (GPUs), all generated from exactly the same code bas
Keywords: Altera OpenCL; Vivado HLS; Vectorization; Loop coarsening; Loop tiling

Modeling and synthesis of communication subsystems for loop accelerator pipelines - Dutta, Hritam; Hannig, Frank; Schmid, Moritz; Keinert, Joachim (Proceedings of the 21st IEEE International Conference on Application-specific Systems, Architectures, and Processors (ASAP), 2010)
The communication synthesis for data transfer and synchronization between loop accelerators is a major challenge in streaming applications. The complexity of the problem arises from the fact that optimal memory mapping and address generation in communication subsystems for parallel data access and out-of-order communication depend on tiling and scheduling choices. This paper solves the problem of communication synthesis by leveraging the windowed synchronous data flow (WSDF) model for communication synthesis. In this context, an intermediate representation of communicating loops in the polyhedral model and a unified methodology for their projection onto the WSDF model is proposed. Finally, we present the architecture template, synthesis methodology, and overhead of the communication primitive. © 2010 IEEE.

Netlist-Level IP Protection by Watermarking for LUT-Based FPGAs - Schmid, Moritz; Ziener, Daniel; Teich, Jürgen (Proceedings of IEEE International Conference on Field-Programmable Technology, 2008)
This paper presents a novel approach to watermark FPGA designs on the netlist level. We restrict the dynamically addressable part of the logic table, thus freeing space for insertion of signature bits into lookup tables (LUTs). In this way, we tightly integrate the watermark with the design so that simply removing mark carrying components would damage the intellectual property core. Converting functional LUTs to LUT-based RAMs or shift registers prevents deletion due to optimization. With this technique, we take watermark carrying components out of the scope of optimization algorithms to achieve complete transparency towards development environments. We can extract the marks from the bitfile of an FPGA. The method was tested on a Xil-inx Virtex-II Pro FPGA and showed low overhead in terms of timing and resources at a reasonable number of watermarked cells. © 2008 IEEE.
UnivIS-Import:2015-04-16:Pub.2008.tech.IMMD.inform.netlis

Power Management Strategies for Serial RapidIO Endpoints in FPGAs - Schmid, Moritz; Hannig, Frank; Teich, Jürgen (Proc. of the IEEE International Field-Programmable Custom Computing Machines Symposium (FCCM), 2012)
We propose a novel data budget-based approach to dynamically control the average power consumption of Serial RapidIO endpoint controllers in FPGAs. The key concept of the approach is to not only perform clock-gating on the FPGA-internal components of the communication controller, but to disable the multi-gigabit transceivers during idle periods. The clock synchronization, inherent to serial interfaces, enables us to omit the often needed periodic link sensing, and only enable the controller according to a predefined schedule to transmit the allocated amount of data during a specific interval. Following this approach, we are able to reduce the dynamic power consumption by up to 77 % on average. © 2012 IEEE.
UnivIS-Import:2015-04-16:Pub.2012.tech.IMMD.inform.powerm

Rapid Prototyping for Hardware Accelerators in the Medical Imaging Domain - Schmid, Moritz (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2015)

Real-Time Range Image Preprocessing on FPGAs - Schmid, Moritz; Blocherer, Markus; Hannig, Frank; Teich, Jürgen (Proc. International Conference on Reconfigurable Computing and FPGAs, 2013)
Unprocessed range images acquired by some range sensing modality often show bumpy surfaces and distorted object boundaries, complicating post processing, such as 3D-registration and feature extraction. The effects are mostly caused by noise due to sensor limitations, but can be mitigated through applying image processing techniques, as for example defect pixel interpolation, bilateral temporal averaging, and edge-preserving noise filtering. In this work, we present an approach to perform preprocessing of range images on field programmable gate arrays (FPGAs) using single precision floating point arithmetic. Moreover, we present an FPGA infrastructure, including image acquisition from a host computer via PCI express. Although this work is based on range images obtained from a Microsoft Kinect range sensing camera, the presented approach is applicable to modalities for range image acquisition, in general. The proposed image processing pipeline can be run at 150MHz for VGA-resolution images and imposes a latency close to 2 ms. © 2013 IEEE.
UnivIS-Import:2015-04-16:Pub.2013.tech.IMMD.inform.realti

Reconfigurable Hardware Generation of Multigrid Solvers with Conjugate Gradient Coarse-Grid Solution - Schmitt, Christian; Schmid, Moritz; Kuckuk, Sebastian; Köstler, Harald; Teich, Jürgen; Hannig, Frank (Parallel Processing Letters, 2018)
Not only in the field of high-performance computing, field-programmable gate arrays (FPGAs) are a soaringly popular accelerator technology. However, they use a completely different programming paradigm and tool set compared to CPUs or even GPUs, adding extra development steps and requiring special knowledge, hindering widespread use in scientific computing. To bridge this programmability gap, domain-specific languages are a popular choice to generate low-level implementations from an abstract algorithm description. In this work, we demonstrate our approach for the generation of numerical solver implementations based on the multigrid method for FPGAs from the same code base that is also used to generate code for CPUs using a hybrid parallelization of MPI and OpenMP. Our approach yields in a hardware design that can compute up to 11 V-cycles per second with an input grid size of 4096x4096 and solution on the coarsest using the conjugate gradient method on a mid-range FPGA, beating vectorized, multi-threaded execution on an Intel Xeon processor.

Robustness Analysis of Watermark Verification Techniques for FPGA Netlist Cores - Ziener, Daniel; Schmid, Moritz; Teich, Jürgen (Design Methodologies for Secure Embedded Systems, 2010)
In this paper we analyze the robustness of watermarking techniques for FPGA IP cores against attacks. Unlike most existing watermarking techniques, the focus of our techniques lies on ease of verification, even if the protected cores are embedded into a product. Moreover, we have concentrated on higher abstraction levels for embedding the watermark, particularly at the logic level, where IP cores are distributed as netlist cores. With the presented watermarking methods, it is possible to watermark IP cores at the logic level and identify them with a high likelihood and in a reproducible way in a purchased product from a company that is suspected to have committed IP fraud. For robustness analysis we enhanced a theoretical watermarking model, originally introduced for multimedia watermarking. Finally, two exemplary watermarking techniques for netlist cores using different verification strategies are described and the robustness against attacks is analyzed. © 2010 Springer-Verlag Berlin Heidelberg.
UnivIS-Import:2015-04-20:Pub.2010.tech.IMMD.inform.robust_0

Synthesis and Optimization of Image Processing Accelerators using Domain Knowledge - Reiche, Oliver; Häublein, Konrad; Reichenbach, Marc; Schmid, Moritz; Hannig, Frank; Teich, Jürgen; Fey, Dietmar (Journal of Systems Architecture, 2015)
In the domain of image processing, often real-time constraints are required. In particular, in safety-critical applications, timing is of utmost importance. A common approach to maintain real-time capabilities is to offload computations to dedicated hardware accelerators, such as Field Programmable Gate Arrays (FPGAs). Designing such architectures is per se already a challenging task, but finding the right design point between achieving as much throughput as necessary while spending as few resources as possible is an even bigger challenge. To address this design challenge in the domain of image processing, several approaches have been presented that introduce an additional layer of abstraction between the developer and the actual target hardware. One approach is to use a Domain-Specific Language (DSL) to generate highly optimized code for synthesis by general purpose High-Level Synthesis (HLS) frameworks. Another approach is to instantiate a generic VHDL IP-Core library for local imaging operators. Elevating the description of image algorithms to such a higher abstraction level can significantly reduce the complexity for designing hardware accelerators targeting FPGAs. We provide a comparison of results for both approaches, a non-expert algorithm developer can achieve. Furthermore, we present an automatic optimization process to give the algorithm developer even more control over trading execution time for resource usage, that could be applied on top of both approaches. To evaluate our optimization procedure, we compare the resulting FPGA accelerators to highly optimized Graphics Processing Unit (GPU) implementations of several image filters relevant for close-to-sensor image and video processing with stringent real-time constraints, such as in the automotive domain.
UnivIS-Import:2016-02-10:Pub.2015.tech.IMMD.IMMD3._synth
Keywords: Hardware Accelerators; Image Processing; Synthesis; Code Generation

System Integration of Tightly-Coupled Processor Arrays using Reconfigurable Buffer Structures - Hannig, Frank; Schmid, Moritz; Lari, Vahid; Boppu, Srinivas; Teich, Jürgen (Proc. ACM International Conference on Computing Frontiers, 2013)
As data locality is a key factor for the acceleration of loop programs on processor arrays, we propose a buffer architecture that can be configured at run-time to select between different schemes for memory access. In addition to traditional address-based memory banks, the buffer architecture can deliver data in a streaming manner to the processing elements of the array, which supports dense and sparse stencil operations. Moreover, to minimize data transfers to the buffers, the design contains an interlinked mode, which is especially targeted at 2-D kernel computations. The buffers can be used individually to achieve high data throughput by utilizing a maximum number of I/O channels to the array, or concatenated to provide higher storage capacity at a reduced amount of I/O channels. Copyright 2013 ACM.
UnivIS-Import:2015-04-16:Pub.2013.tech.IMMD.inform.system
Keywords: Processor array; Reconfigurable buffer; System integration