Rate-Distortion Optimization Based on Two-Pass Encoding for HEVC

Rate-distortion optimization (RDO) is a crucial technique in block-based hybrid video encoders, which determines the coding option for a certain coding unit to achieve optimal rate-distortion (R-D) performance. However, the current RDO, which is implemented into High Efficiency Video Coding (HEVC) test model, i.e., HM, is far from being global optimal as it ignores the interaction among coding options. Recent studies have shown that the dependent RDO methods can improve the coding efficiency of encoders by exploring the R-D dependency among coding units, but these methods do not consider the R-D characteristics of coding units. In this paper, we proposed a two-pass encoding based RDO method, which combines both the R-D dependency and R-D characteristics, to further improve the coding efficiency. First, a frame is encoded with the original HEVC scheme to obtain the R-D model of coding tree units (CTU) and the bit budget of the frame. Second, an optimal equation combines the R-D model and R-D dependency is established to adaptively determine the Lagrange multiplier and quantization parameter (QP) for each CTU. Finally, the current frame is encoded with a new scheme again. Experimental results show that compared with the original HM, the proposed method obtains R-D performance improvements of 5.3% and 5.6% on average under configurations of low-delay B and P frames, respectively.