CMP-based saliency model for stereoscopic omnidirectional images

Abstract Although many saliency models for images had been proposed, the saliency model for stereoscopic omnidirectional image (SOI) is still a very important but not yet deeply studied topic. In addition, one of main disadvantages of the existing saliency models is difficult to simulate the characteristics of a wide field of view (FOV) of SOIs. To solve the problems, this paper proposes a novel cube map projection (CMP) based saliency model for SOIs. Considering that the equi-rectangular projection (ERP) representation of SOI will result in obvious stretch distortion, and what the human eyes view are the viewport images by using head mounted display (HMD), we construct the saliency model of the SOI in its less shape distorted CMP plane. First, the SOI with ERP format is converted into the CMP format. In order to reduce the boundary effect between the faces in CMP plane, its four horizontal faces are moved to obtain the horizontal global face and the horizontal local faces for which the corresponding saliency maps are subsequently computed by combining color distance and depth distance in the graph model. Then, to establish the correlations of these faces, the horizontal face saliency map is calculated by weighting the horizontal global face saliency map to the horizontal local face saliency map. Meanwhile, the vertical face saliency map is defined and obtained. Finally, the horizontal face saliency map and vertical face saliency map are spliced and re-projected to ERP plane to obtain the final saliency map of the ERP-mapped SOI. The experiments on the public ODI database are performed to compare the proposed methods with the state-of-the-art methods, and the results show that the proposed method achieves better performance of estimating SOI saliency map in terms of six well-known quantitative metrics.