An FPGA Avro Parser Generator for Accelerated Data Stream Processing

Hahn T, Schüll D, Wildermann S, Teich J (2023)

Publication Type: Conference contribution, Conference Contribution

Publication year: 2023

Conference Proceedings Title: Proceedings of the 2nd Workshop on Novel Data Management Ideas on Heterogeneous (Co-)Processors (NoDMC)

Event location: Dresden DE

DOI: 10.18420/BTW2023-46

Abstract

Big Data applications frequently involve processing data streams encoded in semi-structured data formats such as JSON, Protobuf, or Avro. A major challenge in accelerating data stream processing on FPGAs is that the parsing of such data formats is usually highly complex. This is especially true for JSON parsing on FPGAs, which lies in the focus of related work. The parsing of the binary Avro format, on the other hand, is perfectly suited for being processed on FPGAs and can thus serve as an enabler for data stream processing on FPGAs. In this realm, we present a methodology for parsing, projection, and selection of Avro objects, which enforces an output format suitable for further processing on the FPGA. Moreover, we provide a generator to automatically create accelerators based on this methodology. The obtained accelerators can achieve significant speedups compared to CPU-based parsers, and at the same time require only very few FPGA resources.  

Authors with CRIS profile

Tobias Hahn Lehrstuhl für Informatik 12 (Hardware-Software-Co-Design) Stefan Wildermann Lehrstuhl für Informatik 12 (Hardware-Software-Co-Design) Jürgen Teich Lehrstuhl für Informatik 12 (Hardware-Software-Co-Design)

Related research project(s)

Query Optimisation and Near-Data Processing on Reconfigurable SoCs for Big Data Analysis (ReProVide) DFG-Schwerpunktprogramm (SPP) 2037 - Skalierbares Datenmanagement für zukünftige Hardware Aug. 28, 2017 - March 1, 2023

How to cite

APA:

Hahn, T., Schüll, D., Wildermann, S., & Teich, J. (2023). An FPGA Avro Parser Generator for Accelerated Data Stream Processing. In Proceedings of the 2nd Workshop on Novel Data Management Ideas on Heterogeneous (Co-)Processors (NoDMC). Dresden, DE.

MLA:

Hahn, Tobias, et al. "An FPGA Avro Parser Generator for Accelerated Data Stream Processing." Proceedings of the 2nd Workshop on Novel Data Management Ideas on Heterogeneous (Co-)Processors (NoDMC), Dresden 2023.

BibTeX: Download