MSEE: Stochastic Cognitive Linguistic Behavior Models for Semantic Sensing

Abstract : This report summarizes the major findings from our research on a semantic information representation framework (SIRF) for visual sensing scenarios. First, the concept and architecture of a cognitive linguistic (CL) based SIRF is introduced. Two levels of information abstraction are proposed within this framework. At the syntactic level, a probabilistic contest free grammar (PCFG) method is employed for information compression and summarization. At the semantic level, a Bayesian network approach is used to achieve semantic concept inference and reasoning. To facilitate the functions of this SIRF, several conceptual primitive modeling methods are proposed, which include a dynamic structure preserving map (DSPM) for individual human action recognition, a Gaussian Process Dynamic Model with Social Network Analysis (GPDM-SNA) for a small human group action recognition, an extended GPDM-SNA method for human object interaction (HOI) recognition, and a pyramid histogram of gradient (pHOG) method for human object recognition based on gait images. In addition to these conceptual primitive models, two quantities sensing modality utility assessment methods are introduced. They are essentially feature selection methods, one is based sparse imputation and one is based on 11 graph. Extensive experiments on publicly available datasets have been conducted to assess the effectiveness of the proposed methods, and highly competitive and promising results have been observed.