Multimodal Interaction for Accessible Smart Homes

Nowadays, houses are being equipped with new smart products and smart sensors, from multiple manufacturers, each offering their own options for interaction with providing varying degrees of usability and user experience. This diverse nature of smart homes and buildings, in general, poses new challenges to interaction design and accessibility. To tackle them, human-building interaction needs to move from articulating different interactive artifacts towards an holistic view of the house as an interactive ecosystem. In a joint effort with Bosch Termotecnologia, S.A., and profiting from recent contributions including an architecture and framework supporting multimodal Interaction, the authors aim to explore novel ways of approaching interaction design with a smart house and proposing smart home applications for all. This paper presents the status of this ongoing work, in the scope of project Smart Green Homes, proposing how multimodal interaction can be supported in the scenario of a smart home and showing first results of tackling human-building interaction through a home assistant serving a family in their daily interactions with the house.