NIC-QF: A design of FPGA based Network Interface Card with Query Filter for big data systems

This paper presents an approach to accelerate query processing on storage and computing separated big data systems. Different from traditional co-processor methods, we propose an FPGA based Network Interface Card with Query Filter (NIC-QF) to pre-filter data on storage nodes. Without modifying the hardware architecture in storage nodes, the traditional NIC can be easily replaced with our NIC-QF to reduce the workload of computing nodes and the corresponding communication overhead. Integrated with the PCIe core, query filter and NIC communicator, NIC-QF can filter the original data on storage nodes and directly send the filtered data to computing nodes of big data systems to reduce the extra communication overheads inside the storage nodes. The filter units in a query filter perform multiple SQL tasks in parallel, and each filter unit is internally pipelined, which further speeds up data processing. The filter units are designed to support general SQL queries in various data formats including TextFile (a row-based storage format) and RCFile (a column-based storage format). Experiments on the TPC-H benchmark and the Tencent data set demonstrate the superiority of our design, saving up to 87.84% of time overhead compared with the traditional approaches.