Long short-term memory network-based user behavior analysis in virtual reality training system—a case study of the ship communication and navigation equipment training

Virtual reality is playing an essential role in the training system. In this paper, we propose a method to analyze the behaviors of users in the VR training system to evaluate their knowledge about the contents and the performance to finish the trained task. First, the user status in the VR environment such as position, orientation, viewpoint, inputs, and timestamp is recorded. Then, we convert these user status data into spatial-temporal semantic trajectories by projecting the viewpoints into the semantic 3D object in the VR system. To speed up the projection and reduce the noises, the 3D targets are semantically generalized. Finally, an LSTM (long short-term memory)-based algorithm is created to classify the spatial-temporal semantic trajectories of user behaviors. The classification can be used to examine if the user is familiar with the training content. According to our experimental results, the proposed method based on the semantic projection can achieve 85% classification accuracy while the direct LSTM-based classification only has 64%.