Visual-based machine understanding framework for decision making on social robot

Social robot acts based on any perceived information to interact and communicate with humans or other robots. One essential type of information is visual-based information captured by camera. Visual data from camera is processed to extract essential information through different perspectives such as gesture detection, facial recognition, sign and object detection, and text understanding. As an intelligent agent, the social robot should be able to determine and perform socially rational action autonomously based on the processed information. In this paper, a general framework of visual-based information process to produce rational action is presented. The proposed architecture is implemented on a platform of humanoid robot as a test subject.