Conversational Co-Speech Gesture Generation via Modeling Dialog Intention, Emotion, and Context with Diffusion Models
0
Citation
24
Reference
10
Related Paper
Abstract:
Audio-driven co-speech human gesture generation has made remarkable advancements recently. However, most previous works only focus on single person audio-driven gesture generation. We aim at solving the problem of conversational co-speech gesture generation that considers multiple participants in a conversation, which is a novel and challenging task due to the difficulty of simultaneously incorporating semantic information and other relevant features from both the primary speaker and the interlocutor. To this end, we propose CoDiffuseGesture, a diffusion model-based approach for speech-driven interaction gesture generation via modeling bilateral conversational intention, emotion, and semantic context. Our method synthesizes appropriate interactive, speech-matched, high-quality gestures for conversational motions through the intention perception module and emotion reasoning module at the sentence level by a pretrained language model. Experimental results demonstrate the promising performance of the proposed method.Keywords:
Dialog system
Context model
Topics:
AI dialog systems are now developing from question-answering systems toward advising systems. This includes:- structuring dialog- understanding and generating a wider range of speech acts than simply information request and answer- user modellingUser modelling in HAM-ANS is closely connected to dialog structure and dialog strategy. In advising the user, the system generates and verbalizes speech acts. The choice of the speech act is guided by the user profile and the dialog strategy of the system.
Dialog system
Structuring
Cite
Citations (11)
The emergence of smart devices and recent advances in spoken language technology are currently extending the use of conversational interfaces and spoken interaction to perform many tasks. The dialog management task of a conversational interface consists of selecting the next system response considering the user's actions, the dialog history, and the results of accessing the data repositories. In this paper we describe a dialog management technique adapted to multi-task conversational systems. In our proposal, specialized dialog models are used to deal with each specific subtask of dialog objective for which the dialog system has been designed. The practical application of the proposed technique to develop a dialog system acting as a customer support service shows that the use of these specialized dialog models increases the quality and number of successful interactions with the system in comparison with developing a single dialog model.
Dialog system
Interface (matter)
Cite
Citations (14)
Dialog system
Cite
Citations (115)
Many of current spoken dialog systems can conduct non-task-oriented dialog. The systems that can improve user impression are required for users to keep using them. This paper focuses on self-disclosure, that is a process that a person reveals information about herself/himself to an interlocutor in human-human conversation. It is known that the self-disclosure plays a vital role to develop an intimate relationship. However, it is still unclear how exchanging the self-disclosures affects the user impression in the human-machine dialog. In this paper, we conduct dialog experiments to investigate the effectiveness of mutual self-disclosures between the user and the system. To achieve this goal, we built a spoken dialog system which conducts the dialog that the user and the system disclose information about themselves alternately. The dialog experiments revealed that the proposed system can improve the user impression regarding satisfaction and friendliness.
Dialog system
Impression
User Satisfaction
Cite
Citations (4)
Dialog system
Cite
Citations (1)
Recently, spoken dialog systems using speech recognition technology have been becoming popular. Such as chat-like dialog systems, these systems which do not have any specific purpose are called "Non-Task-oriented spoken dialog system". In this study, we focused on non-task-oriented spoken dialog systems. We have developed a multi-party chat-like dialog system with different preferences between two agents. In order to provide more enjoyable chatting to users, we focused on the relationship between two agents. In this study, we dealt with two types of the chat-like dialog system with two agents: an "Unsympathetic dialog system "where one agent's preference differs from another agent, and a "Sympathetic dialog system" where both agents have the same preference. We evaluated user's degree of satisfaction by using our developed systems. As a result of the experiment, the sympathetic dialog system achieved a better impression in chatting. In order to solve the system problems obtained from the experimental results, we designed a new dialog system in order to deal with a more enjoyable conversation.
Dialog system
Cite
Citations (2)
Dialog system
Cite
Citations (0)
Dialog system
Cite
Citations (2)
Dialog system
Utterance
Cite
Citations (9)
This paper describes a dialog system allowing a human user and a personal assistant agent to communicate through natural language. The Assistant uses explicit task representation and knowledge representation to achieve an intelligent dialog. The dialog architecture is independent from the knowledge or task representation and includes different models: dialog models, task models, user models and domain models. The paper describes how the dialog manager uses the models to conduct the dialog and how the approach is related to other approaches in the same domain. The system has been implemented in a collaborative environment in order to personalize services and to improve interaction.
Dialog system
Representation
Cite
Citations (2)