Computer Engineering and Applications ›› 2022, Vol. 58 ›› Issue (5): 225-231.DOI: 10.3778/j.issn.1002-8331.2010-0235

• Pattern Recognition and Artificial Intelligence • Previous Articles     Next Articles

Fingertip Detection Model Based on UGF-Net

LIU Jia, BIAN Fangzhou, CHEN Dapeng, LI Weibin   

  1. B-DAT&CICAEET, School of Automation, Nanjing University of Information Science & Technology, Nanjing 210044, China
  • Online:2022-03-01 Published:2022-03-01



  1. 南京信息工程大学 自动化学院 B-DAT&CICAEET,南京 210044

Abstract: In the field of human-computer interaction, accurate fingertip detection has a great impact on the richness and flexibility of interaction. However, due to the small size of fingertip, accurate and robust fingertip detection is still a challenging task. In order to improve the accuracy of fingertip detection, this paper proposes a fingertip detection model based on depth convolution neural network UGF-Net(unified-gesture-and-fingertip- network). The model can be used for fingertip detection and gesture recognition at the same time. The YOLO algorithm is used to extract the gesture area, and the FCNN output visual Gaussian heat map is used to realize fingertip detection. Finally, the effectiveness and robustness of the proposed fingertip detection model are verified by experiments. The model is tested on the SCUT-Ego-Gesture data set. The results show that the accuracy of fingertip detection can reach 99.8%, and the average frame rate of real-time video image is 34.5 frame/s, which meets the requirements of real-time.

Key words: convolutional neural network, deep learning, fingertip detection, gesture recognition, target detection

摘要: 在人机交互领域,精确的人手指尖检测对交互的丰富度、灵活度有很大影响。然而,由于指尖的尺寸较小,精确、鲁棒的指尖检测目前仍然是一项颇具挑战性的任务。为了提升指尖检测的准确率与实时性,提出一种基于深度卷积神经网络的指尖检测模型UGF-Net(unified-gesture-and-fingertip-network)。该模型可以同时进行指尖检测与手势识别,利用YOLO算法来提取手势区域,通过FCNN输出可视化高斯热图来实现指尖检测。通过实验验证该指尖检测模型的有效性和鲁棒性,在SCUT-Ego-Gesture数据集上对模型进行了测试,结果表明,指尖检测的准确率可达到99.8%,且实时视频图像的平均帧率达到34.5?frame/s,满足实时性的要求。

关键词: 卷积神经网络, 深度学习, 指尖检测, 手势识别, 目标检测