Juan Cao

Communication University of China

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Pengzhou Zhang

Communication University of China

Qiang He

Communication University of China

Wenchao Song

Communication University of China

Shaoyang Yuan

Communication University of China

Haiyan Lu

University of Chinese Academy of Sciences

Zhaohong Liu

Northeast Normal University

Yujun Wen

Communication University of China

Peng Zhou Zhang

Communication University of China

Rui Zhen

Communication University of China

Junpeng Gong

Communication University of China

Cooperative Institutions

C-Com Satellite Systems (Canada)

304

Chinese Academy of Sciences

Shanghai Jiao Tong University

University of Chinese Academy of Sciences

Sun Yat-sen University

Communication University of China

Wuhan University

Zhejiang University

Chinese Academy of Medical Sciences & Peking Union Medical College

Sichuan University

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

Research on the Application of Virtual Human Synthesis Technology in Human-Computer Interaction

Rui Zhen Wenchao Song Juan Cao

With the rapid development of 5g, artificial intelligence, chip technology (GPU computing power), and capital investment, human body synthesis technology is entering a growth stage. According to people's comprehensive image, we divide them into idol school, academic school, and practical school; According to the driving methods of virtual human synthesis, we divide it into avatar type, voice or text content driving type, and human-computer interaction type. We recently introduced the relevant model of the synthesis method and the implementation process of virtual human lip synthesis and explained the synthesis process. We discussed voice2face, the latest scheme for real-time synthesis of virtual human, but this method still can not synthesize high-quality video in milliseconds.

Avatar

Virtual actor

View synthesis

10.1109/icis54925.2022.9882355

Cite

Citations (3)

Generating Natural Language Descriptions From Tables

IEEE Access (2020)

Juan Cao

This paper proposes a neural generative architecture, namely NLDT, to generate a natural language short text describing a table which has formal structure and valuable information. Specifically, the architecture maps fields and values of a table to continuous vectors and then generates a natural language description by leveraging the semantics of a table. The NLDT architecture adopts a two-level neural model to make the most of the structure of a table to fully express the relationship between contents. To deal with the problem of out-of-vocabulary, this paper develops a simple and fast word-conversion method that replaces rare words appearing in texts with common field information in tables and directly replicates contents from table to the output sequence according to the field information. Besides, this paper adds the concept of theme to adapt the NLDT architecture to open domain and improves beam search algorithm to strengthen the results in the inference stage. On the WEATHERGOV dataset, the NLDT architecture improves the state-of-the-art BLEU-4 score from 61.01 to 62.89 and the current state-of-the-art F1 score from 73.21 to 78. On the WIKIBIO and WIKITABLE datasets, the NLDT architecture achieves a BLEU-4 score of 45.77 and 38.71 respectively which also outperform the state-of-the-art approaches. Furthermore, this paper introduces a Chinese dataset WIKIBIOCN including 33,244 biographies with corresponding tables. On the WIKIBIOCN dataset, the NLDT architecture achieves a BLEU-4 score of 38.87 and fairly good manual evaluation.

Table (database)

10.1109/access.2020.2979115

Cite

Citations (12)

Human-Computer Interaction System: A Survey of Talking-Head Generation

Electronics (2023)

Rui Zhen Wenchao Song Qiang He Juan Cao Lei Shi

Virtual human is widely employed in various industries, including personal assistance, intelligent customer service, and online education, thanks to the rapid development of artificial intelligence. An anthropomorphic digital human can quickly contact people and enhance user experience in human–computer interaction. Hence, we design the human–computer interaction system framework, which includes speech recognition, text-to-speech, dialogue systems, and virtual human generation. Next, we classify the model of talking-head video generation by the virtual human deep generation framework. Meanwhile, we systematically review the past five years’ worth of technological advancements and trends in talking-head video generation, highlight the critical works and summarize the dataset.

Human interaction

Interactive voice response

10.3390/electronics12010218

Cite

Citations (31)

Multi-Level Feature Dynamic Fusion Neural Radiance Fields for Audio-Driven Talking Head Generation

Applied Sciences (2025)

Wenchao Song Qiong Liu Yanchao Liu Pengzhou Zhang Juan Cao

Audio-driven cross-modal talking head generation has experienced significant advancement in the last several years, and it aims to generate a talking head video that corresponds to a given audio sequence. Out of these approaches, the NeRF-based method can generate videos featuring a specific person with more natural motion compared to the one-shot methods. However, previous approaches failed to distinguish the importance of different regions, resulting in the loss of information-rich region features. To alleviate the problem and improve video quality, we propose MLDF-NeRF, an end-to-end method for talking head generation, which can achieve better vector representation through multi-level feature dynamic fusion. Specifically, we designed two modules in MLDF-NeRF to enhance the cross-modal mapping ability between audio and different facial regions. We initially developed a multi-level tri-plane hash representation that uses three sets of tri-plane hash networks with varying resolutions of limitation to capture the dynamic information of the face more accurately. Then, we introduce the idea of multi-head attention and design an efficient audio-visual fusion module that explicitly fuses audio features with image features from different planes, thereby improving the mapping between audio features and spatial information. Meanwhile, the design helps to minimize interference from facial areas unrelated to audio, thereby improving the overall quality of the representation. The quantitative and qualitative results indicate that our proposed method can effectively generate talk heads with natural actions and realistic details. Compared with previous methods, it performs better in terms of image quality, lip sync, and other aspects.

Feature (linguistics)

10.3390/app15010479

Cite

Citations (0)

Multi-channel Live Video Processing Method Based on Cloud-edge Collaboration

Zhen Guo Pengzhou Zhang Junjie Xia Zhou Zhe Juan Cao

The rise of 5G edge computing technology brings new development opportunities for video applications. In this paper, a multi-channel video analysis method based on cloud-edge collaboration is designed. Considering edge computing resources and network transmission time, a cloud-edge collaborative multi-channel live video processing framework is proposed. The proposed method makes full use of the processing advantages of edge computing and the cloud-edge collaboration, which can effectively meet the needs of efficient computing and network transmission for live video streaming. This paper analyzes the cloud-edge collaborative multi-channel video processing process in details. Finally, this paper conducts the test comparison between 4G and 5G live network environment, and the deployment verification in 5G network environment. The effectiveness of the cloud-edge collaborative multi-channel video solution in the 5G environment is verified through the test.

Edge device

10.1109/trustcom56396.2022.00198

Cite

Citations (1)

A knowledge graph-based content selection model for data-driven text generation

International Journal of Reasoning-based Intelligent Systems (2017)

Jun Gong Juan Cao Peng Zhou Zhang

Content selection is a critical task for natural language generation. A novel approach based on knowledge graph is proposed. Structure data is mapping to the graph and combined with user defined knowledge. The model analyses the content selection features on the graph, and automatically learns the content selection rules. The model was evaluated in the domain of weather forecasting.

Knowledge graph

Content (measure theory)

Text generation

10.1504/ijris.2017.10011231

Cite

Citations (1)

Multimodal emotion recognition in conversation based on speaker dependency

Jiayi Zhang Shaoyang Yuan Haiyan Lu Juan Cao

10.1117/12.3051017

Cite

Citations (0)

A General Construction Method of Virtual Simulation Experiment Platform Based on Bibliometrics and Analytic Hierarchy Process

Education Sciences (2023)

Keyun Zhu Juan Cao Guowei Chen Qiang He Pengzhou Zhang

Virtual simulation can solve the challenges of high cost, long cycle time, and inaccessibility in traditional experimental teaching, which is far-reaching for talent training. This study combines bibliometric visualization theory with AHP (Analytic Hierarchy Process). It establishes a hierarchical evaluation model of a virtual simulation experimental teaching platform based on 842 questionnaires and 4787 articles, including 68,306 citation records, and deconstructing the complex evaluation problem into several multidimensional factors by attributes and relationships. Based on this, a virtual simulation experimental teaching platform construction scheme for IP protocol analysis based on a network covert communication perspective is outputted, which is compatible with the research results. The experimental platform takes a task-driven teaching method as the core, mainly including four modules of context creation, task determination, independent learning, and effect evaluation. The experience of building this platform can be extended to other disciplines, leading the teaching reform exploration of practice-based, innovation-focused, and engineering-critical, helping to implement the flipped classroom, and promoting the development of education modernization.

10.3390/educsci13010080

Cite

Citations (6)

Dynamic Region Fusion Neural Radiance Fields for Audio-Driven Talking Head Generation

Qiang He Juan Cao Haiyan Lu Pengzhou Zhang

10.1109/mlnlp63328.2024.10799951

Cite

Citations (0)

ERNCLS: A Cross-Lingual Text Summarization Model Integrating Entity Relation Extraction Task

Zhaohong Liu Juan Cao Jia Yu Guowei Chen Qiang He

The current research on generative cross-lingual summarization models mainly adopts the framework of seq2seq, where entity relation errors are inevitable. How to reduce the occurrence of errors and improve the accuracy of summary generation needs to be addressed urgently. In this regard, we try to add the task of extracting entity relations as an auxiliary task in the Chinese-to-English cross-lingual text summarization model to reduce the entity relation errors in the summaries, and propose two fusion strategies for merging the auxiliary task. In addition, we also adjust the dataset for this task. The experiment proves that it is effective to reduce errors as well as improve the performance of cross-lingual text summarization model by merging the auxiliary task of extracting entity relations in the way of splicing summaries and entity relation triples as real labels of training sets.

Relationship extraction

Grammaticality

10.1109/iciba56860.2023.10165547

Cite

Citations (0)