Computer Engineering and Applications ›› 2023, Vol. 59 ›› Issue (14): 192-200.DOI: 10.3778/j.issn.1002-8331.2204-0084

• Graphics and Image Processing • Previous Articles     Next Articles

Real-Time Hand Detection Method Based on Lightweight Network

JIN Fangrui, WANG Yangping, YONG Jiu   

  1. 1.School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
    2.Gansu Artificial Intelligence and Graphics and Image Processing Engineering Research Center, Lanzhou 730070, China
  • Online:2023-07-15 Published:2023-07-15

基于轻量化网络的实时手部检测方法

靳芳蕊,王阳萍,雍玖   

  1. 1.兰州交通大学 电子与信息工程学院,兰州 730070
    2.甘肃省人工智能与图形图像处理工程研究中心,兰州 730070

Abstract: Aiming at the hand detection method based on deep learning, the network model has complex structure, slow detection speed and serious memory consumption, which is difficult to meet the needs of real-time human-computer interaction at the mobile terminal, a lightweight network model is proposed to realize the real-time detection of hand. The model is based on SSD. Firstly, the lightweight network MobileNetV2 is used as the backbone feature extraction network of the model to reduce the amount of parameters and computational complexity of the model. Then, aiming at the problem of insufficient feature extraction, a multi-scale convolution and feature fusion module is proposed and applied to three prediction feature layers to increase the adaptability of the network to different scale features by connecting convolution kernels of different sizes. Finally, the candidate box acquisition method is improved. The [K]-means++ clustering algorithm is used to adaptively generate the candidate box suitable for the hand, and the hand is accurately located to improve the detection accuracy of the model. In order to verify the effectiveness of the proposed method, relevant experiments are carried out on two public data sets, Ego Hands and Oxford Hand. The experimental results show that the accuracy of this method on the two data sets is 96.56% and 73.56% respectively, the detection speed is 45.4 FPS and 41.2 FPS, and the memory occupied by the model is only 19.5 MB. Finally, the algorithm is deployed on the mobile terminal for testing, and the results show that this method can provide a lightweight method for hand detection.

Key words: lightweight model, hand detection, SSD, feature enhancement, mobile terminal

摘要: 针对基于深度学习的手部检测方法网络模型结构复杂,检测速度慢,内存消耗严重,难以满足移动端进行实时人机交互的需求,提出一种轻量化网络模型实现对手部的实时检测。该模型以SSD为基础结构,将轻量化网络MobileNetV2作为模型主干特征提取网络,减小模型的参数量和计算复杂度;针对特征提取不充分问题,提出多尺度卷积和特征融合模块,并应用在其中三个预测特征层,通过连接不同尺寸的卷积核,增加网络对不同尺度特征的适应性;改进候选框获取方式,采用[K]-means++聚类算法自适应生成适合手部的候选框,对手部进行准确定位来提高模型的检测精度。为验证所提方法的有效性,在Ego Hands、Oxford Hand两个公开数据集上进行相关实验。实验结果显示,该方法在两个数据集上的准确率分别达96.56%和73.56%,检测速度达45.4?FPS和41.2?FPS,且模型所占内存仅为19.5?MB。最后将算法部署在移动端进行测试,结果表明该方法能够为手部检测提供一个轻量化的方法。

关键词: 轻量化模型, 手部检测, SSD, 特征增强, 移动终端