Homogeneous Multimodal Access to the Digital Home for People with Cognitive Disabilities

This paper introduces a multimodal dialogue system, which facilitates access to the digital home for people who suffer from cognitive disabilities. The user interface is implemented on a smartphone and allows interaction via speech and pointing gestures. A consistent control concept and a meaningful graphical design allow an intuitive handling of the device. The ISO standard URC enables the system to supervise and control heterogeneous home devices. One focal point is collaborative problem solving which supports users with memory deficits in planning and pursuing objectives. Furthermore, the reaction of the system is adapted to user and interaction context.