Apollo SignSound: an intelligent system applied to ubiquitous healthcare of deaf people

In Brazil, there are more than 9.3 million people with Hearing Impairment (PWHI) and deaf who face daily accessibility difficulties. On the other hand, there is the growth of the use of mobile devices and the application of the Internet of Things. The motivation for the development of this work lies in the absence of user interface systems based on sign language and customizable according to the profile that is applied to the prevention of risks external to the health of the deaf. This paper proposes the Apollo SignSound system, which promotes accessibility for PWHI and deaf people in a smart home environment, especially regarding safety. The scientific contribution of Apollo SignSound lies in the detection of ambient risks using neural networks. Besides this, SignSound also considers the user profile, mainly the degree of deafness, to generate accessible notifications represented in Brazilian Sign Language (LIBRAS in Portuguese). The notifications of risk sent to the smartphone of PWHI or deaf also can vibrate or turn on the light of the device. We implemented a prototype of a smart home that collects environmental sounds and notifies the deaf user. The scenario-based assessment included three activities of daily living events of a deaf user: a kettle boiling over the stove, a dog barking, and one person knocking on the door. The results indicate the means of the f score of 0.73 for accuracy evaluation. Usability and acceptance evaluations were performed by five deaf users and the results indicate the approval of 90% in the perceived ease of use and perceived usefulness.