Hotspots Integrating of Expert and Beginner Experiences of Machine Operations through Egocentric Vision

Beginners can often provide useful information regarding machine operations, e.g., how difficult a specific operation is or how beginners deal with difficulties. However, their experiences oftentimes widely vary, and it is difficult to summarize these diverse experiences in an extensive way. To try and solve this problem, this study focused on developing a framework for integrating beginners' and experts' experiences into a unified operation model. A baseline model was first obtained based on hand-machine interactions automatically extracted from experts' egocentric vision records. Then, the beginners' behaviors were integrated by dynamically aligning them to the baseline model. Through this process, an integrated model based on the experiences of a wide range of users was achieved. We applied our method to the operation experiences of two tabletop devices, an IH heater and a sewing machine. The results show good potentials in modeling the common and different behaviors among experts and beginners.