Follow, Select and Assemble Method FSAM for VE and RE Assembly

ABSTRACT We modified the methods by Ballard et al. [1]. They sought to study eye-hand coordination strategies and created both a real world and a virtual model copying task consisting of three different areas: model, source, and workspace. Participants followed a pickup and place exercise and used a mouse to control stimuli presented on a computer monitor in the virtual task. We also considered the method presented by Hayhoe et al. [2] and Aivar et al. [3] who designed a similar model copying task in a 3D virtual environment. Stimuli were displayed in a head mounted display and participants held a motion sensor to select and move virtual objects in 3D space. Moreover, Aivar et al. [3] also included extra assembly pieces and a variation in the position of the assembly pieces located in the resource area. • It proposes an assembly task designed at a 1:1 scale for two environments, real and virtual environments. • It introduces a reading sequence for the model that is being replicated and it also introduces distractor assembly blocks with similar colors and shapes as the required assembly blocks, and a change in the location for all assembly blocks in the resource area. • It modifies the interaction for the VR environment by using hand gestures to select, move and position virtual assembly blocks. This was possible by incorporating a LEAP® motion controller which although it does not provide haptic feedback, it provides a virtual representation of the participant's hand. Our VE also includes visual and auditory feedback to guide depth perception and virtual control. The software used for this research study is available at: http://virtualete.com/research/fsam.php