Fast single depth intra mode decision for depth map coding in 3D-HEVC
11
Citation
9
Reference
10
Related Paper
Citation Trend
Abstract:
In 3D-HEVC, single depth intra mode has been applied and has been integrated into depth intra skip mode for efficient depth map coding. With single depth intra mode, one 2N×2N prediction unit (PU) is predicted without high computational prediction process. In this paper, we propose a fast single depth intra mode decision method to address the problem of high computational complexity burden in depth intra mode decision of 3D-HEVC. To remove unnecessary computational complexity at the encoder, we early decide single depth intra mode for pruning quadtree in 3D-HEVC. This paper characterizes the statistics of smooth depth map signals for depth intra modes and analyzes distortion metrics of view synthesis optimization functionality as a decision criterion. With this proposed criterion, a single depth intra mode for intra coding has been detected and hierarchical CU/PU selection for intra coding can be stopped in 3D-HEVC. As a consequence, it utilizes the correlation between hierarchical block-based video coding and coding unit (CU)/PU mode decision for depth map coding so that a large number of recursive rate-distortion cost calculations can be skipped. We demonstrate the effectiveness of our approach experimentally. The simulation results show that the proposed scheme can achieve approximately 25.6% encoding time saving with 0.07% video PSNR/total bitrate gain and 0.18% synthesized view PSNR/total bitrate loss under all intra configuration.Keywords:
Depth map
Quadtree
Algorithmic efficiency
View synthesis
Macroblock
Multi-view depth estimation is an important part in the virtual view synthesis and depth map coding within the 3DV/FTV applications. However, existing depth estimation methods generally have difficulty to generate good depth results around object edges or in areas with less texture. To solve this problem, we propose an improved algorithm to obtain a smooth and accurate depth map for view synthesis in 3D video coding. The proposed method aims to preserve details in the depth map and simultaneously smooth depths in object boundary occluded area, less texture and discontinuity regions, to avoid blurring in the synthesized views. Experimental results show a significant improvement of the initial input depth maps and coding efficiency, as well as a reduction of view synthesis artifacts.
View synthesis
Depth map
Texture Synthesis
Algorithmic efficiency
Discontinuity (linguistics)
Cite
Citations (1)
Depth map
View synthesis
Algorithmic efficiency
Cite
Citations (16)
This paper presents a method that jointly performs stereo matching and inter-view interpolation to obtain the depth map and virtual view image. A novel view synthesis method based on depth map layers representation of the stereo image pairs is proposed. The main idea of this approach is to separate the depth map into several layers of depth based on the disparity distance of the corresponding points. The novel view synthesis can be interpolated independently to each layer of depth by masking the particular depth layer. The final novel view synthesis obtained with the entire layers flattens into a single layer. Since the image view synthesis is performed in separate layers, the extracted new virtual object can be superimposed onto another 3D scene. The method is useful for free viewpoint video application with a small number of camera configurations. Based on the experimental results, it shows that the algorithm could improve the efficiency of finding the depth map and to synthesis the new virtual view images.
View synthesis
Depth map
Interpolation
Representation
Virtual image
Cite
Citations (10)
Depth information indicates the distance of an object in the three dimensional (3D) scene from the camera view-point, typically represented by eight bits. Since the depth map is useful in various multimedia applications, such as three dimensional television (3DTV) and free-viewpoint television (FTV), we need to acquire a single or multi-view depth maps and process them effectively. In this paper, we propose a new coding scheme for multi-view depth video data using depth view synthesis. We first apply a 3D warping method to synthesize a virtual depth image for the current view using the multi-view depth information. We also propose a hole filling method to compensate for the holes generated during the depth map synthesis process. Finally, we utilize the synthesized depth map for the current view as an additional reference frame in encoding the current depth map. Experimental results show that the proposed algorithm achieves approximately 0.69 dB of PSNR gain on average, compared to JMVM 1.0.
View synthesis
Depth map
Image warping
Cite
Citations (15)
High Efficiency Video Coding (HEVC) is the latest video coding standard, which adapts quadtree structure based Coding Tree Unit (CTU) to improve the coding efficiency. In HEVC encoding process, the CTU is recursively partitioned into coding units according to the quadtree depth. This technique increases the coding efficiency of HEVC, however, the achieved coding efficiency comes at the cost of high computational complexity. In this paper, we propose a fast C-TU quadtree depth decision algorithm to reduce the computational complexity of HEVC. Firstly, based on the best C-TU depth correlation among spatial and temporal neighboring CTUs, an early quadtree depth 0 decision algorithm is proposed. Then, according to the correlation between the prediction unit mode and the best CTU depth selection, a quadtree depth 3 skipped decision algorithm is proposed. Experimental results show that the proposed algorithm can achieve 40% on average encoding time saving, while maintaining a comparable rate-distortion performance.
Quadtree
Algorithmic efficiency
Cite
Citations (14)
This paper presents a method that jointly performs stereo matching and inter-view interpolation to obtain the depth map and virtual view image. A novel view synthesis method based on depth map layers representation of the stereo image pairs is proposed. The main idea of this approach is to separate the depth map into several layers of depth based on the disparity distance of the corresponding points. The novel view synthesis can be interpolated independently to each layer of depth by masking the particular depth layer. The final novel view synthesis obtained with the entire layers flattens into a single layer. Since the image view synthesis is performed in separate layers, the extractednew virtual object can be superimposed onto another 3D scene. The method is useful to imply free viewpoint video application with a small number of camera configurations. Based on the experimental results, it shows that the algorithm improve the efficiency of finding the depth map and to synthesis the new virtual view images.
View synthesis
Depth map
Interpolation
Representation
Virtual image
Cite
Citations (1)
Multi-view video plus depth (MVD) is a 3D video representation. In MVD, the depth map provides the scene distance information and is used to render the virtual view through Depth Image Based Rendering (DIBR) technique. The depth map coding error will induce distortion in the rendered virtual views. This paper proposes a mathematic model that can estimate the synthesized virtual view distortion induced by depth map compression, and the model is employed to the rate distortion optimization (RDO) in the depth map coding. Based on the rendered virtual view quality, a Lagrangian optimization adjustment scheme at Coding Unit (CU) level is proposed to improve the depth map encoding efficiency. Experimental results demonstrate that the proposed method can improve the BD-PSNR of virtual view for 0.62 dB, and the encoding complexity reduces compared with the view synthesis optimization (VSO) technique in the 3D-HEVC Test Model (HTM).
View synthesis
Depth map
Distortion (music)
Rate–distortion optimization
Cite
Citations (2)
Accurate and high-quality depth maps are required in lots of 3D applications, such as multi-view rendering, 3D reconstruction and 3DTV. However, the resolution of captured depth image is much lower than that of its corresponding color image, which affects its application performance. In this paper, we propose a novel depth map super-resolution (SR) method by taking view synthesis quality into account. The proposed approach mainly includes two technical contributions. First, since the captured low-resolution (LR) depth map may be corrupted by noise and occlusion, we propose a credibility based multi-view depth maps fusion strategy, which considers the view synthesis quality and interview correlation, to refine the LR depth map. Second, we propose a view synthesis quality based trilateral depth-map up-sampling method, which considers depth smoothness, texture similarity and view synthesis quality in the up-sampling filter. Experimental results demonstrate that the proposed method outperforms state-of-the-art depth SR methods for both super-resolved depth maps and synthesized views. Furthermore, the proposed method is robust to noise and achieves promising results under noise-corruption conditions.
View synthesis
Depth map
Texture Synthesis
Cite
Citations (56)
H.264 is a new international video coding standard that can achieve considerably higher coding efficiency than conventional standards. Its coding gain has been achieved by employing advanced video coding methods. Specially, the increased number of macroblock modes and the complex mode decision procedure using the Lagrangian optimization are the main factors for increasing coding efficiency. Although H.264 obtains improved coding efficiency, it is difficult to do an real-time encoding because it considers all coding parameters in the mode decision procedure. In this paper, we propose a fast mode decision algorithm which classifies the macroblock modes in order to determine the optimal mode having low complexity quickly. Simulation results show that the proposed algorithm can reduce the encoding time by 34.95% on average without significant PSNR degradation or bit-rate increment. In addition, in order to show the validity of simulation results, we set up a low boundary condition for coding efficiency and complexity and show that the proposed algorithm satisfies the low boundary condition.
Macroblock
Algorithmic efficiency
Decision model
Cite
Citations (0)
The performance of view synthesis using depth image based rendering (DIBR) highly depends on the accuracy of depth map. Inaccurate boundary alignment between texture image and depth map especially for large depth discontinuities always cause annoying artifacts in disocclusion regions of the synthesized view. Pre-filtering approach and reliability-based approach have been proposed to tackle this problem. However, pre-filtering approach blurs the depth map with drawback of degradation of the depth map and may also cause distortion in non-hole region. Reliability-based approach uses reliable warping information from other views to fill up holes and is not suitable for the view synthesis with single texture video such as video-plus-depth based DIBR applications. This paper presents a simple and efficient depth map preprocessing method with use of texture edge information to refine depth pixels around the large depth discontinuities. The refined depth map can make the whole texture edge pixels assigned with foreground depth values. It can significantly improve the quality of the synthesized view by avoiding incorrect use of foreground texture information in hole filling. The experimental results show the proposed method achieves superior performance for view synthesis by DIBR especially for large baseline.
View synthesis
Depth map
Classification of discontinuities
Image warping
Distortion (music)
Cite
Citations (15)