Abstract: This paper proposes a method to realize a speech-to-gesture conversion for communication between speech impediments and healthy people. The keyword spotting is employed to recognize the key words from input speech signals. At the same time, the three dimensional gesture models of keywords are built by 3D modeling technology according to the “Chinese sign language”. The speech-to-gesture conversion is finally realized by playing the corresponding 3D gestures with OpenGL from the results of keyword spotting. Tests show that the realized keyword spotting achieves 90.1% of average recognition rate on letters and numbers. The converted gestures obtain 4.4 of mean opinion score. Therefore the proposed method can be applied to the communications between normal persons and speech impediments.

Key words: keyword spotting, gesture modeling, speech to gesture conversion

摘要： This paper proposes a method to realize a speech-to-gesture conversion for communication between speech impediments and healthy people. The keyword spotting is employed to recognize the key words from input speech signals. At the same time, the three dimensional gesture models of keywords are built by 3D modeling technology according to the “Chinese sign language”. The speech-to-gesture conversion is finally realized by playing the corresponding 3D gestures with OpenGL from the results of keyword spotting. Tests show that the realized keyword spotting achieves 90.1% of average recognition rate on letters and numbers. The converted gestures obtain 4.4 of mean opinion score. Therefore the proposed method can be applied to the communications between normal persons and speech impediments.

关键词: keyword spotting, gesture modeling, speech to gesture conversion