FLICS: FPGA-acceLerated Ingestion of documents into Columnar Storage

Hahn T, Wildermann S, Teich J (2026)

Publication Type: Conference contribution, Conference Contribution

Publication year: 2026

Event location: Mainz DE

Abstract

The increasing demand for efficient data processing has led to the development of various data storage formats and processing techniques. Columnar storage formats, such as Apache Parquet, are widely used for their efficiency in handling analytical queries on nested documents. However, incoming data is often encoded in row-based formats, necessitating a transformation into a columnar format before analytical queries can be applied.
This transformation is computationally intensive and can introduce significant resource and performance overheads. As a remedy, we introduce FLICS, an FPGA-accelerated solution for parsing and ingesting row-oriented Avro data into the columnar Parquet storage format. FLICS leverages an application-specific instruction-set processor (ASIP) architecture to parse Avro data and convert it into the columnar format.
The ASIP is controlled by an instruction program tailored to parse a specific schema. Consequently, any schema modification merely necessitates loading a new instruction sequence into the instruction memory. 
Additionally, we introduce a schema compiler that automatically generates these instruction programs, facilitating seamless schema transitions. Our evaluation demonstrates that FLICS achieves up to 3.3 times higher throughput compared to state-of-the-art software solutions running on a fully utilized 24-core CPU, while using only a small fraction of the available FPGA resources.

Authors with CRIS profile

Tobias Hahn Lehrstuhl für Informatik 12 (Hardware-Software-Co-Design) Stefan Wildermann Lehrstuhl für Informatik 12 (Hardware-Software-Co-Design) Jürgen Teich Lehrstuhl für Informatik 12 (Hardware-Software-Co-Design)

Related research project(s)

Query Optimisation and Near-Data Processing on Reconfigurable SoCs for Big Data Analysis (ReProVide) DFG-Schwerpunktprogramm (SPP) 2037 - Skalierbares Datenmanagement für zukünftige Hardware Aug. 28, 2017 - March 1, 2023

How to cite

APA:

Hahn, T., Wildermann, S., & Teich, J. (2026). FLICS: FPGA-acceLerated Ingestion of documents into Columnar Storage. In Proceedings of the 39th GI/ITG International Conference on Architecture of Computing Systems. Mainz, DE.

MLA:

Hahn, Tobias, Stefan Wildermann, and Jürgen Teich. "FLICS: FPGA-acceLerated Ingestion of documents into Columnar Storage." Proceedings of the 39th GI/ITG International Conference on Architecture of Computing Systems, Mainz 2026.

BibTeX: Download