Computer Engineering and Applications ›› 2021, Vol. 57 ›› Issue (3): 156-161.DOI: 10.3778/j.issn.1002-8331.1910-0429

Previous Articles     Next Articles

Application of Improved SSD Algorithm in Chinese Sign Language Recognition

ZHOU Zhou, HAN Fang, WANG Zhijie   

  1. School of Information Science and Technology, Donghua University, Shanghai 201620, China
  • Online:2021-02-01 Published:2021-01-29



  1. 东华大学 信息科学与技术学院,上海 201620


The sign language recognition technology based on computer vision can bring great convenience to bilingual teaching in schools. In recent years, with the rapid development of deep learning technology, the accuracy and detection speed of sign language recognition has also been greatly improved. Different from the method of using color marking and external technology(such as Kinect’s palm positioning technology), an improved SSD(Single-Shot Multibox Detector) network is proposed to perform target detection on gestures to complete Chinese sign language recognition. For the small target of the hand, SE-Net is embedded in the feature layer of SSD for channel weight distribution, the loss function is improved to better resolve the imbalance of positive and negative sample, and data argument is carried out by mixup. Then the recognition result will be matched with the sign language key gesture template library of China. Experiments verify that the proposed algorithm has higher recognition rate and speed in sign language recognition.

Key words: Chinese sign language recognition, object detection, channel weight distribution, data enhancement


基于计算机视觉的手语识别技术能为聋校双语教学带来很大的便利。近年来,随着深度学习技术的蓬勃发展,手语识别的准确率和速度有了极大的提高。与使用颜色标记和外界技术(如Kinect手心定位技术)的方法不同,提出一种改进的SSD(Single-Shot Multibox Detector)网络,对手势进行目标检测完成中国手语识别。针对手部小目标,将SE-Net嵌入SSD中的特征层进行通道权重分配,改进损失函数更好地应对正负样本不均衡问题,使用mixup进行数据增强,将手势识别结果在中国手语关键手势模板库中进行匹配,从而完成动态手语识别。实验证明,该算法在手语识别上具有较高的准确率和识别速度。

关键词: 中国手语识别, 目标检测, 通道权重分配, 数据增强