An FPGA-accelerated high-throughput data optimization system for high-speed transfer via wide area network

In this paper, we propose an FPGA-accelerated data optimization system for high-speed data transfer via Wide Area Network (WAN). To maintain high quality in cloud services, high-speed transfer among data centers via WAN is important. To accelerate transfers via WAN, data optimization techniques are used to reduce the size of the data transmitted to the WAN. Compression and deduplication are widely-used data optimization techniques; data sent is shrunk before transmission and restored upon receipt. However, the computational time of these techniques is so great that it is difficult to achieve high performance using only a CPU. To solve this problem, we developed an efficient data optimization system using both CPU and FPGA. We designed a dedicated accelerator whose architecture is suitable for each processing characteristic of data optimization. We also proposed an efficient data flow inside the FPGA and communication method between the CPU and the FPGA. Thanks to the appropriate pipelining and parallelization, and the data flow, the measured throughput of our developed accelerator achieves over 41 Gbps. With this accelerator, we can realize a data transfer system whose throughput is 40 Gbps end-to-end: as far as we know the fastest performance for a WAN optimization system.