Sign Language to Speech Translation

Although a certain fraction of the world suffers from speech and hearing disabilities, sign language is not a widespread language across the world. In today's oral world, a gesture-based language is not particularly popular with the general masses. However, sign language itself is a fully developed language with multiple regional dialects across the globe. Therefore, to aid with a smoother communication between the speaking and non-speaking world, technical developments can be introduced. A considerable amount of work has been done in this direction. The basic need of the hour is for an application that can function in real-time and can facilitate real-time conversations between a person who can sign in sign language and one that cannot. This paper proposes an application that works on this problem statement. In order to construct such an application designed to respond in real-time and aid a live conversation between a speaking and nonspeaking individual, it is necessary to allow for live video inputs to be made to the app which would then be translated to speech. This paper proposes the use of convolutional neural networks (CNN) alongside the use of Text-to-Speech translator. By use of the CNN algorithm, the gestures can be identified by the proposed application and converted to text, which can then be converted into speech.