Color-Sensitivity-based Rate-Distortion Optimization for H.265/HEVC

Rate-Distortion Optimization (RDO) is an important step in video coding to achieve the best quality under a certain compression ratio constraint. The traditional RDO assigns equal importance to different color components. However, Human Visual System (HVS) has different sensitivities to different components. In this paper, the color-sensitivity-based combined PSNR (CSPSNR) is utilized as the distortion measurement in the process of RDO, where the characteristics of the color sensitivities of HVS are taken into account. Firstly, the distortion weights of luma and chroma components are derived from the criterion of maximizing CSPSNR. Then Lagrange multiplier and quantization parameter (QP) are adjusted according to the variation of distortion weights among different components. Finally, the CSPSNR-based RDO (CSRDO) adaptively calculates the RD costs of luma and chroma components under different sampling rates to improve the coding efficiency of the whole sequence. Experimental results in H.265/HEVC demonstrate that the proposed method can achieve 3.11% and 3.58% BD-RATE gain for AI and RA configurations in terms of CSPSNR on average.