HVUAN – A Rapid-Development Framework for Spanish-Speaking Virtual Humans

We present a rapid-development framework for Spanish-speaking virtual humans (VH). Our approach tries to minimize the effort of working with VH systems while preserving customizability of the interaction. We effortlessly generate a 3D representation of the VH and blendshapes for lipsync and facial expression using MakeHuman and a Blender plugin. We animate the VH using captured motion data using a Kinect and affordable motion capture software. The framework uses Cortana for Windows 10 to handle Spanish speech recognition and Excel and/or yEd to specify the dialog interaction in graph format, where nodes contain agent, audio, and animation information and edges hold grammar options for Cortana. The system can handle recognition while it speaks, allowing for interrupts, and automatically generates gestures or speech when only text is specified. This system should work for any language for which Cortana supplies a speech recognizer.