Multimodal interaction in process control rooms: are we there yet?

Although process control rooms are advanced digital environments with a multitude of desktop and large displays, there still exists a gap between the interaction technologies being employed in pervasive displays used in other settings and those used in control rooms. Previous research on large displays is extensively geared towards investigating social interaction in non-serious environments. The potential of employing multimodal interactive displays to support work and collaboration in process control rooms is underexplored. This paper explores modern control rooms as interactive collaborative environments and presents a multimodal prototype that promotes the users' freedom and flexibility of work through gestural and spoken interaction. The design of the prototype was informed by three feedback sessions and two user studies with various stakeholders. Our findings indicate multimodal interaction in process control rooms should (a) support the feeling of control and safety during the interaction, (b) enable efficient access to information provided by system from a distance, and (c) provide freedom and flexibility of use anywhere in the control room.