Timed Petri Nets for Multimodal Interaction Modeling

Humans naturally use the multiple modalities of speech, gesture, and gaze when they communicate. They also engage in turn-taking to manage the seizing and yielding of the speaking oor, a process that controls the execution of turns comprising discrete speech, gesture, and gaze events. We are interested in modeling such interaction processes for a social robot to use that can be transferred between dierent domains. Timed Petri nets (TPNs) are currently uncommon in human-robot interaction (HRI) but oer an attractive representation for modeling concurrency and synchronization when controlling behavior for multiple modalities. Their representation also permits the intuitive and modular combination of rule-based behavior expression with statistical timing models. We describe their utility in application to multimodal interaction and compare them to nite state