VESS: An Unstructured Data-Oriented Storage System for Multi-Disciplined Virtual Experiment Platform

The maintenance of unstructured data for a large data-related system is still a hard nut to crack, especially in the multi-disciplined virtual experiment platform (MDVEP), where lots of unstructured data are produced, small in size and coming from various applications. This paper presents a new storage system named VESS for unstructured data from MDVEP, which can improve the performance of the system and simplify the development of upper-layer applications. It provides HTTP-based data request method as system user-friendly interface. Index table relying on a universal metadata file is used to ensure the consistency and integrity of replicas. Message queue is adopted to sequence modification operations and guarantee consistency, while signature-based authentication guarantees the security. Experimental results show that VESS can serve as a data storage service for virtual experiment with high performance.