Derma: Silent Speech Interaction Using Transcutaneous Motion Sensing

Silent speech interaction (SSI) enables speech communication without uttering an actual voice and can have the potential to make the speech available in public places. However, lip-reading SSI requires a camera in front of the face and thus it is not suitable for mobile use. Ultrasound-SSI requires expensive and complex equipment. The proposed Derma is a much simpler method by skin motion sensing. Two 6-DOF accelerometer/angular velocity sensors attached under the chin acquire 12-dimensional multidimensional information of skin motion caused by the utterance. With the neural network, 35 silent commands are identified with a recognition rate of 94%. While previous SSI studies have normally trained with vocal speech, we also propose a learning method from non-vocal speech only. The proposed solution is less visible, lightweight, and is not affected by lighting conditions.



  • 暦本純一、西村悠、Derma: 皮膚運動計測によるサイレントスピーチインタラクション、情報処理学会インタラクション2020
  • Jun Rekimoto, Yu Nishimra, Derma: Silent Speech Interaction Using Transcutaneous Motion Sensing, Augmented Humans 2021
Both comments and trackbacks are currently closed.