Evaluation of high-frequency financial transaction processing in distributed memory systems

A financial system requires processing a large-scale data stream. These data are considerably large for realtime processing. Using a distributed memory system is the most efficient means to process and store this data. In addition, using a high-performance interconnection technology such as InfiniBand is necessary because existing Ethernet technology that uses low bandwidth and possesses high latency can have detrimental effects on the financial system. In particular, in a system that processes large-scale financial data, high latency can cause financial loss. Therefore, high-performance interconnection technology is critical for processing large-scale financial data. We propose a scheme that uses high-performance interconnection technology in a distributed memory system for processing a large-scale data stream. In experiments that test the performance of our system, we use Redis, which is an open source data structure server, and messages of the size of the actual financial data in existence. Experimental results show that our distributed memory system using InfiniBand can process financial transactions with high throughput and low latency.