Human-computer interaction experiments in an immersive virtual reality environment for e-learning applications

The present work concerns a Human-Computer Interaction (HCI) experiment in an Immersive Virtual Reality (IVR) environment. The user is located at the focal point of a four metre hemispherical dome projection screen. By combining a 5DT data glove with an iPhone 3G, the user can directly interact with a computer program/client. The visualisation is implemented with the Unity 3D 2.5 multiplatform game development tool. The client communicates with a .NET socket server allowing continuous data streaming from connected sensors, i.e., finger flexures, hand gestures (from the data glove) and orientation data (from the iPhone's accelerometer sensors). The main goal of the experiment is to evaluate interactive techniques for users who are training in an IVR environment. An IVR application is demonstrated to highlight the value of the proposed method in an intuitive e-learning experience associated with the teleoperation of underground coal mining equipment.