Digital twin for human-robot interactive welding and welder behavior analysis

This paper presents an innovative investigation on prototyping a digital twin (DT) as the platform for human-robot interactive welding and welder behavior analysis. This human-robot interaction (HRI) working style helps to enhance human users' operational productivity and comfort; while data-driven welder behavior analysis benefits to further novice welder training. This HRI system includes three modules: 1) a human user who demonstrates the welding operations offsite with her/his operations recorded by the motion-tracked handles; 2) a robot that executes the demonstrated welding operations to complete the physical welding tasks onsite; 3) a DT system that is developed based on virtual reality (VR) as a digital replica of the physical human-robot interactive welding environment. The DT system bridges a human user and robot through a bi-directional information flow: a) transmitting demonstrated welding operations in VR to the robot in the physical environment; b) displaying the physical welding scenes to human users in VR. Compared to existing DT systems reported in the literatures, the developed one provides better capability in engaging human users in interacting with welding scenes, through an augmented VR. To verify the effectiveness, six welders, skilled with certain manual welding training and unskilled without any training, tested the system by completing the same welding job; three skilled welders produce satisfied welded workpieces, while the other three unskilled do not. A data-driven approach as a combination of fast Fourier transform (FFT), principal component analysis (PCA), and support vector machine (SVM) is developed to analyze their behaviors. Given an operation sequence, i.e., motion speed sequence of the welding torch, frequency features are firstly extracted by FFT and then reduced in dimension through PCA, which are finally routed into SVM for classification. The trained model demonstrates a 94.44% classification accuracy in the testing dataset. The successful pattern recognition in skilled welder operations should benefit to accelerate novice welder training.