HIVE: A cross-platform, modular visualization framework for large-scale data sets

Abstract Leading-edge supercomputers, such as the K computer and Fugaku, have been designed to achieve the highest computational performance possible as well as to tackle “Grand Challenge” class of simulations with unprecedented scale. This significant increase in the simulation scale has directly imposed a pressure on the entire end-to-end simulation workflow, which includes the pre- and post-processing such as the visualization. During the simulation code development and refinement process in such HPC environment, a variety of auxiliary computational systems with different hardware and software configurations can be employed for the post-processing activities. Therefore, a visualization application capable of running on such heterogeneous hardware environment, which uses common visualization pipeline workflow and unified abstract representation becomes highly valuable. In this paper, we present a visualization framework, named HIVE (Heterogeneously Integrated Visual-analytics Environment), designed to meet these requirements by using lightweight and cross-platform Lua scripting language for describing the desired visualization pipeline workflow, which was named as “Visualization Scene” script. Different visualization pipeline functionality modules such as data loading, rendering, and image compositing written in C/C++ programming language can be utilized via Lua by using its binding functionality. HIVE has currently integrated some cross-platform modules, and is capable of running on different hardware systems, ranging from x86 laptops to SPARC64 based supercomputers with tens of thousands of processors. As a future direction, we expect to include the supercomputers using Arm-based Fujitsu A64FX CPU such as the Fugaku, which is under installation, and other commercial systems from Fujitsu and Cray.