Query Optimisation and Near-Data Processing on Reconfigurable SoCs for Big Data Analysis (Phase II) (ReProVide II INF6)
Third party funded individual grant
Acronym: ReProVide II INF6
Start date : 01.08.2021
End date : 31.07.2024
Website: https://www.dfg-spp2037.de/me943-9/
Project details
Scientific Abstract
Analysing petabytes of data in an affordable amount of time and energy requires massively parallel processing of data at their source. Active research is therefore directed towards emerging hardware architectures to reduce the volume of data close to their source and towards sound query analysis and optimisation techniques to exploit such novel architectures. The goal of the ReProVide project is to investigate FPGA-based solutions for smart storage and near-data processing together with novel query-optimisation techniques that exploit the speed and reconfigurability of FPGA hardware for a scalable and powerful (pre-)filtering of Big Data.
In the first funding phase, we have fostered the fundamentals for this endeavour. In particular, we have designed an FPGA-based PSoC architecture of so-called Reconfigurable Data Provider Units (RPUs). For data processing and filtering, an RPU exploits the capabilities of dynamic (run-time) hardware reconfiguration of modern FPGAs to load pre-designed hardware accelerators on-the-fly. An RPU is able to process SQL queries or parts of them in hardware in combination with CPU cores also available on the PSoC. For the integration of RPUs into a DBMS, new cost models had to be developed, taking the capabilities and characteristics of an RPU into account. Here, we have elaborated a novel hierarchical (multi-level) query optimisation to determine which operations are worthwhile to be assigned to a RPU (query partitioning) and how to deploy and execute the assigned (sub-)queries or database operators on the RPU (query placement). The implemented query optimiser shares the work between the global optimiser of the DBMS (in our case Apache Calcite) and an architecture-specific local optimiser running on the RPU.
In the second funding phase, our major research goals will be:
1.) Stream processing: RPUs could equally be beneficial for the filtering of streams. Here, a plethora of fundamentally new module functionality will have to be investigated to support non-standard operators, leading to RPUs applicable to a much more diverse class of tasks including window operations and data-preparation functionality.
2.) Scalability: User interaction with modern databases usually involves not only one, but a sequence of queries. At the same time, multiple applications are running concurrently. Here, we will design an eight-node RPU cluster attached to storage and network to enable the distributed and parallel data processing of large databases and data streams. Also required are concepts for data partitioning and novel query optimisation techniques, making use of query-sequence information.
3.) Demonstrator & Evaluation: As a testbed and a proof of the benefits of the ReProVide approach in general and an FPGA-based RPU cluster in particular, we want to analytically as well as experimentally evaluate the margins of energy reductions that become possible through near-data processing.
