Evaluating Index Systems of High Energy Physics

Nowadays, more and more scientific data has been produced through high-energy physics (HEP) facilities. Even in one particle physics experiment, the generated data reaches to petabytes scale. Retrieving data from massive data occupies a large proportion of data processing in HEP. Hence, the data query latency and throughput are the most important metrics for HEP data management. Inspired by the indexing technology of databases, the technology that improves the performance of data retrieval through the HEP data indexing, becomes the mainstream in the HEP data management. In this paper, focusing on two typical index systems–MySQL and HBase–for HEP data management, which are the typical SQL and NoSQL system respectively, we evaluate them from the perspectives of overall performance, system and micro-architecture behaviors. We find that HBase achieves higher performance than MySQL with the data scale increasing.