A Distortion Estimation Method Using Integer Operations in H.264/AVC Encoder

In this paper, a new low-complexity distortion estimation method for H.264 rate-distortion optimized mode decision is proposed. The coding processes, such as DCT, quantization, inverse quantization, inverse DCT, and reconstruction are needed to compute the distortion in an H.264 encoder. To reduce these processes, we estimate distortion using integer operations with coefficients obtained in the quantization process. Inverse quantization, inverse DCT, and reconstruction processes are not needed by the proposed method. For quantization parameters 24 to 36, experimental results show that the time saving of rate-distortion optimized mode decision is on average 29 % and as high as 42 % with negligible degradation in coding performance.