Chuanbo Tang

University of Science and Technology of China

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Xihua Sheng

University of Science and Technology of China

Zhuoyuan Li

Nanjing University of Information Science and Technology

Haotian Zhang

Hong Kong Polytechnic University

Dong Liu

University of Science and Technology of China

Li Li

University of Science and Technology of China

Li Li

Shandong University of Finance and Economics

Junqi Liao

University of Science and Technology of China

Dong Liu

Yunnan Investment Group (China)

Yuqi Li

Zhejiang University of Technology

Feng Wu

Nanjing University of Science and Technology

Cooperative Institutions

University of Science and Technology of China

Chinese Academy of Sciences

Beijing Institute of Technology

Tsinghua University

Fudan University

University of Science and Technology Beijing

École Polytechnique Fédérale de Lausanne

Microsoft Research Asia (China)

Northeastern University

Huazhong University of Science and Technology

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

USTC-TD: A Test Dataset and Benchmark for Image and Video Coding in 2020s

arXiv (Cornell University) (2024)

Zhuoyuan Li Junqi Liao Chuanbo Tang Haotian Zhang Yuqi Li

Image/video coding has been a remarkable research area for both academia and industry for many years. Testing datasets, especially high-quality image/video datasets are desirable for the justified evaluation of coding-related research, practical applications, and standardization activities. We put forward a test dataset namely USTC-TD, which has been successfully adopted in the practical end-to-end image/video coding challenge of the IEEE International Conference on Visual Communications and Image Processing in 2022 and 2023. USTC-TD contains 40 images at 4K spatial resolution and 10 video sequences at 1080p spatial resolution, featuring various content due to the diverse environmental factors (scene type, texture, motion, view) and the designed imaging factors (illumination, shadow, lens). We quantitatively evaluate USTC-TD on different image/video features (spatial, temporal, color, lightness), and compare it with the previous image/video test datasets, which verifies the wider coverage and more diversity of the proposed dataset. We also evaluate both classic standardized and recent learned image/video coding schemes on USTC-TD with PSNR and MS-SSIM, and provide an extensive benchmark for the evaluated schemes. Based on the characteristics and specific design of the proposed test dataset, we analyze the benchmark performance and shed light on the future research and development of image/video coding. All the data are released online: https://esakak.github.io/USTC-TD.

Benchmark (surveying)

10.48550/arxiv.2409.08481

Cite

Citations (0)

NVC-1B: A Large Neural Video Coding Model

arXiv (Cornell University) (2024)

Xihua Sheng Chuanbo Tang Li Li Dong Liu Feng Wu

The emerging large models have achieved notable progress in the fields of natural language processing and computer vision. However, large models for neural video coding are still unexplored. In this paper, we try to explore how to build a large neural video coding model. Based on a small baseline model, we gradually scale up the model sizes of its different coding parts, including the motion encoder-decoder, motion entropy model, contextual encoder-decoder, contextual entropy model, and temporal context mining module, and analyze the influence of model sizes on video compression performance. Then, we explore to use different architectures, including CNN, mixed CNN-Transformer, and Transformer architectures, to implement the neural video coding model and analyze the influence of model architectures on video compression performance. Based on our exploration results, we design the first neural video coding model with more than 1 billion parameters -- NVC-1B. Experimental results show that our proposed large model achieves a significant video compression performance improvement over the small baseline model, and represents the state-of-the-art compression efficiency. We anticipate large models may bring up the video coding technologies to the next level.

10.48550/arxiv.2407.19402

Cite

Citations (0)

Offline and Online Optical Flow Enhancement for Deep Video Compression

arXiv (Cornell University) (2023)

Chuanbo Tang Xihua Sheng Zhuoyuan Li Haotian Zhang Li Li

Video compression relies heavily on exploiting the temporal redundancy between video frames, which is usually achieved by estimating and using the motion information. The motion information is represented as optical flows in most of the existing deep video compression networks. Indeed, these networks often adopt pre-trained optical flow estimation networks for motion estimation. The optical flows, however, may be less suitable for video compression due to the following two factors. First, the optical flow estimation networks were trained to perform inter-frame prediction as accurately as possible, but the optical flows themselves may cost too many bits to encode. Second, the optical flow estimation networks were trained on synthetic data, and may not generalize well enough to real-world videos. We address the twofold limitations by enhancing the optical flows in two stages: offline and online. In the offline stage, we fine-tune a trained optical flow estimation network with the motion information provided by a traditional (non-deep) video compression scheme, e.g. H.266/VVC, as we believe the motion information of H.266/VVC achieves a better rate-distortion trade-off. In the online stage, we further optimize the latent features of the optical flows with a gradient descent-based algorithm for the video to be compressed, so as to enhance the adaptivity of the optical flows. We conduct experiments on a state-of-the-art deep video compression scheme, DCVC. Experimental results demonstrate that the proposed offline and online enhancement together achieves on average 12.8% bitrate saving on the tested videos, without increasing the model or computational complexity of the decoder side.

Optical Flow

10.48550/arxiv.2307.05092

Cite

Citations (0)

Offline and Online Optical Flow Enhancement for Deep Video Compression

Proceedings of the AAAI Conference on Artificial Intelligence (2024)

Chuanbo Tang Xihua Sheng Zhuoyuan Li Haotian Zhang Li Li

Video compression relies heavily on exploiting the temporal redundancy between video frames, which is usually achieved by estimating and using the motion information. The motion information is represented as optical flows in most of the existing deep video compression networks. Indeed, these networks often adopt pre-trained optical flow estimation networks for motion estimation. The optical flows, however, may be less suitable for video compression due to the following two factors. First, the optical flow estimation networks were trained to perform inter-frame prediction as accurately as possible, but the optical flows themselves may cost too many bits to encode. Second, the optical flow estimation networks were trained on synthetic data, and may not generalize well enough to real-world videos. We address the twofold limitations by enhancing the optical flows in two stages: offline and online. In the offline stage, we fine-tune a trained optical flow estimation network with the motion information provided by a traditional (non-deep) video compression scheme, e.g. H.266/VVC, as we believe the motion information of H.266/VVC achieves a better rate-distortion trade-off. In the online stage, we further optimize the latent features of the optical flows with a gradient descent-based algorithm for the video to be compressed, so as to enhance the adaptivity of the optical flows. We conduct experiments on two state-of-the-art deep video compression schemes, DCVC and DCVC-DC. Experimental results demonstrate that the proposed offline and online enhancement together achieves on average 13.4% bitrate saving for DCVC and 4.1% bitrate saving for DCVC-DC on the tested videos, without increasing the model or computational complexity of the decoder side.

Optical Flow

Online video

10.1609/aaai.v38i6.28317

Cite

Citations (3)