关键词: 人机交互, 手势识别, 深度学习, 关键点检测, 幻影模块

Abstract:

Human-Computer Interaction（HCI） is mostly realized by keyboard and mouse at present. The interactive algorithm based on deep learning gesture recognition is relatively low in accuracy, real-time performance and system stability. This paper proposes a novel ghost machine gesture interaction system improved on lightweight OpenPose. Through lightweight OpenPose, human hand is modeled into 21 key points. Then on the basis of MobileNetV1, Part Affinity Fields（PAF） method is applied to detect key points of human hand and draw a simple skeleton diagram. To further improve HCI’s real-time performance, Ghost Module is used to reduce the dimension of convolution layer, and the same recognition effect is obtained with fewer hardware resources. Finally, the verification environment is set up, and the pattern matching is carried out according to the drawn skeleton diagram of human hand. The interactive control instructions are generated based on the matching recognition results and then are transmitted to the Arduino UNO platform through Bluetooth communication to control the car to realize interactive response. After the initial training, the system achieves an accuracy rate of 58.7% on COCO2017 verification set, which maintains the recognition effect of the original OpenPose network and lightweight OpenPose network, and it realizes the speed of 32-36 frames per second and high gesture recognition rate on the household PC.

Key words: human-computer interaction, gesture recognition, deep learning, key point detection, ghost module