Joint Separable and Non-Separable Transforms for Next-Generation Video Coding

Throughout the past few decades, the separable discrete cosine transform (DCT), particularly the DCT type II, has been widely used in image and video compression. It is well-known that, under first-order stationary Markov conditions, DCT is an efficient approximation of the optimal Karhunen–Loeve transform. However, for natural image and video sources, the adaptivity of a single separable transform with fixed core is rather limited for the highly dynamic image statistics, e.g., textures and arbitrarily directed edges. It is also known that non-separable transforms can achieve better compression efficiency for images with directional texture patterns, yet they are computationally complex, especially when the transform size is large. In order to achieve higher transform coding gains with relatively low-complexity implementations, we propose a joint separable and non-separable transform. The proposed separable primary transform, named enhanced multiple transform (EMT), applies multiple transform cores from a pre-defined subset of sinusoidal transforms, and the transform selection is signaled in a joint block level manner. Moreover, a non-separable secondary transform (NSST) method is proposed to operate in conjunction with EMT. Unlike the existing non-separable transform schemes which require excessive amounts of memory and computation, the proposed NSST efficiently improves coding gain with much lower complexity. Extensive experimental results show that the proposed methods, in a state-of-the-art video codec, such as high efficiency video coding, can provide significant coding gains (average 6.9% and 4.5% bitrate reductions for intra and random-access coding, respectively).