Computer Engineering and Applications ›› 2021, Vol. 57 ›› Issue (16): 159-166.

### Improved Ghost Machine Gesture Interaction System Based on Lightweight OpenPose

TAN Lixing, LU Jiaqi, ZHANG Xiaonan, LIU Yuhong, ZHANG Rongfen

1. Big Data and Information Engineering Institute, Guizhou University, Guizhou 550023, China
• Online:2021-08-15 Published:2021-08-16

### 基于轻量级OpenPose改进的幻影机手势交互系统

1. 贵州大学 大数据与信息工程学院，贵州 550023

Abstract:

Human-Computer Interaction（HCI） is mostly realized by keyboard and mouse at present. The interactive algorithm based on deep learning gesture recognition is relatively low in accuracy, real-time performance and system stability. This paper proposes a novel ghost machine gesture interaction system improved on lightweight OpenPose. Through lightweight OpenPose, human hand is modeled into 21 key points. Then on the basis of MobileNetV1, Part Affinity Fields（PAF） method is applied to detect key points of human hand and draw a simple skeleton diagram. To further improve HCI’s real-time performance, Ghost Module is used to reduce the dimension of convolution layer, and the same recognition effect is obtained with fewer hardware resources. Finally, the verification environment is set up, and the pattern matching is carried out according to the drawn skeleton diagram of human hand. The interactive control instructions are generated based on the matching recognition results and then are transmitted to the Arduino UNO platform through Bluetooth communication to control the car to realize interactive response. After the initial training, the system achieves an accuracy rate of 58.7% on COCO2017 verification set, which maintains the recognition effect of the original OpenPose network and lightweight OpenPose network, and it realizes the speed of 32-36 frames per second and high gesture recognition rate on the household PC.