TieLent: A Casual Neck-Mounted Mouth Capturing Device for Silent Speech Interaction

With the increased use of smart speakers, silent speech interaction (SSI) is attracting attention. Unfortunately, traditional silent speech interaction methods require the addition of obtrusive sensors and devices around the user's face, making wearability and portability a challenge. Considering that most uses for smart speakers do not require many words, we suggest a more casual approach, TieLent, which can easily be worn between the neck and the chest. TieLent's RGB camera is set away from the user's face, presenting less interference with the user. Although TieLent's camera is not able to capture the whole mouth, when combined with our image-to-speech neural network model, it is able to generate the recognizable speech of 15 commands with an average accuracy of 94%.