Review of Deep Learning Methods for Palm Vein Recognition

doi:10.3778/j.issn.1002-8331.2306-0168

Abstract

Abstract: Palm vein recognition, as a new infrared biometrics technology, has become one of the research hotspots in the field of biometric recognition because of its advantages of high security and liveness detection. In recent years, a great deal of research in this field has promoted the development of palm vein recognition technology by introducing deep learning methods. In order to grasp the latest research status and development direction in the field of palm vein recognition, data acquisition and the mainstream algorithms of data pre-processing are classified and summarized, and the latest progress of palm vein recognition based on deep learning is classified and elaborated in terms of palm vein feature representation, network design and optimization, and lightweight network. In view of the bottleneck of single-modal recognition, the correlation algorithms of multimodal and multi-feature fusion recognition are analyzed and compared. The difficulties and challenges of current research on palm vein recognition are discussed, and the future development trends are prospected and summarized.

Key words: palm vein recognition, deep learning, multimodal fusion

摘要： 掌静脉识别作为一种新兴的红外生物识别技术，因其高安全性、活体检测性等优势已成为当前生物特征识别领域中的研究热点之一。近年来，该领域的大量研究通过引入深度学习方法推动了掌静脉识别技术的发展。为了掌握掌静脉识别领域最新研究现状及发展方向，对数据采集和数据预处理的主流算法进行了分类和总结，并针对基于深度学习的掌静脉识别的最新进展按照掌脉特征表征、网络设计与优化、轻量级网络进行了分类和详细阐述。针对当前单模态识别达到瓶颈等问题，分析并对比了多模态和多特征融合识别相关算法；探讨了当前掌静脉识别的研究难点挑战，并对未来的发展趋势进行了展望与总结。

关键词: 掌静脉识别, 深度学习, 多模态融合

TAN Zhenlin, LIU Ziliang, HUANG Aiquan, CHEN Huihui, ZHONG Yong. Review of Deep Learning Methods for Palm Vein Recognition[J]. Computer Engineering and Applications, 2024, 60(6): 55-67.

谭振林, 刘子良, 黄蔼权, 陈荟慧, 钟勇. 掌静脉识别的深度学习方法综述[J]. 计算机工程与应用, 2024, 60(6): 55-67.

References

[1] MINAEE S, ABDOLRASHIDI A, SU H, et al. Biometrics recognition using deep learning: a survey[J].arXiv:1912. 00271, 2019.
[2] MACGREGOR P, WELFORD R. Veincheck imaging forsecurity[EB/OL].(1991).http: //www.scrip.org/journal/atricles.aspx.
[3] HASSAN N F, ABDULRAZZAQ H I. Pose invariant palm vein identification system using convolutional neural network[J]. Baghdad Science Journal, 2018, 15(4): 503-510.
[4] ZHONG D X, SHAO H K, DU X F. A hand-based multi-biometrics via deep hashing network and biometric graph matching[J]. IEEE Transactions on Information Forensics and Security, 2019, 14(12): 3140-3150.
[5] HORNG S J, VU D T, NGUYEN T V, et al. Recognizing palm vein in smartphones using RGB images[J]. IEEE Transactions on Industrial Informatics, 2022, 18(9): 5992-6002.
[6] SALAZAR-JURADO E H, HERNANDEZ-GARCIA R, VILCHES-PONCE K, et al. Towards the generation of synthetic images of palm vein patterns: a review[J]. arXiv:2205. 10179, 2022.
[7] JIA W, XIA W, ZHANG B, et al. A survey on dorsal hand vein biometrics[J]. Pattern Recognition, 2021, 120: 108-122.
[8] 钟德星, 朱劲松, 杜学峰. 掌纹识别研究进展综述[J]. 模式识别与人工智能, 2019, 32(5): 436-445.
ZHONG D X, ZHU J S, DU X F. Progress of palmprint recognition: a review[J]. Pattern Recognition and Artificial Intelligence, 2019, 32(5): 436-445.
[9] 孙哲南, 赫然, 王亮, 等. 生物特征识别学科发展报告[J]. 中国图象图形学报, 2021, 26(6): 1254-1329.
SUN Z N, HE R, WANG L, et al. Overview of biometrics research[J].Journal of Image and Graphics, 2021, 26(6): 1254-1329.
[10] HAO Y, SUN Z N, TAN T N, et al. Multispectral palm image fusion for accurate contact-free palmprint recognition[C]//Proceedings of IEEE International Conference on Image Processing, 2008: 281-284.
[11] TOME P, MARCEL S. On the vulnerability of palm vein recognition to spoofing attacks[C]//Proceedings of International Conference on Biometrics, 2015: 319-325.
[12] ZHANG D, GUO Z H, LU G M, et al. An online system of multispectral palmprint verification[J]. IEEE Transactions on Instrumentation and Measurement, 2010, 59(2): 480-490.
[13] KABACINSKI R, KOWALSKI M. Vein pattern database and benchmark results[J]. Electronics Letters, 2011, 47(20): 1127.
[14] ZHANG L, CHENG Z X, SHEN Y, et al. Palmprint and palmvein recognition based on DCNN and a new large-Scale contactless palmvein dataset[J]. Symmetry, 2018, 10(4): 78.
[15] TOYGAR O, BABALOLA F O, BITIRIM Y. FYO: a novel multimodal vein database with palmar, dorsal and wrist biometrics[J]. IEEE Access, 2020, 8: 82461-82470.
[16] WU W, WANG Q, YU S Q, et al. Outside box and contactless palm vein recognition based on a wavelet denoising ResNet[J]. IEEE Access, 2021, 9: 82471-82484.
[17] JHINN W L, MICHAEL G K O, CONNIE T, et al. Preliminary work on rotation-invariant algorithms for contactless palm vein biometrics[C]//Proceedings of the International Conference on Information and Communication Technology (ICoICT), 2015: 568-573.
[18] MICHAEL G K O, CONNIE T, TEOH BENG JIN A. An innovative contactless palm print and knuckle print recognition system[J]. Pattern Recognition Letters, 2010, 31(12): 1708-1719.
[19] OUESLATI A, FEDDAOUI N, BELGHITH S, et al. An efficient palm vein region of interest extraction method[C]//Proceedings of the 5th International Conference on Advanced Technologies for Signal and Image Processing, 2020: 1-7.
[20] OTSU N. A threshold selection method from gray-level histograms[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1): 62-66.
[21] ONG MICHAEL G K, CONNIE T, JIN TEOH A B. Touch-less palm print biometrics: novel design and implementation[J]. Image and Vision Computing, 2008, 26(12): 1551-1560.
[22] LIN S, XU T Y, YIN X Y. Region of interest extraction for palmprint and palm vein recognition[C]//Proceedings the 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics, 2016: 538-542.
[23] CHEN Y Y, HSIA C H, CHEN P H. Contactless multispectral palm-vein recognition with lightweight convolutional neural network[J]. IEEE Access, 2021, 9: 149796-149806.
[24] SUN X W, XU Q S, WANG C Y, et al. ROI extraction for online touchless palm vein based on concavity analysis[C]//Proceedings of the 32nd Youth Academic Annual Conference of Chinese Association of Automation, 2017: 1123-1126.
[25] ITO K, SATO T, AOYAMA S, et al. Palm region extraction for contactless palmprint recognition[C]//Proceedings of International Conference on Biometrics, 2015: 334-340.
[26] AYKUT M, EKINCI M. AAM-based palm segmentation in unrestricted backgrounds and various postures for palmprint recognition[J]. Pattern Recognition Letters, 2013, 34(9): 955-962.
[27] 吴凯, 沈文忠, 贾丁丁, 等.融合Transformer和CNN的手掌静脉识别网络[J].计算机工程与应用, ?2023, 59(24): 98-109.
WU K, SHEN W Z, JIA D D, et al. Palm vein recognition network combining Transformer and CNN[J]. Computer Engineering and Applications, 2023, 59(24): 98-109.
[28] ZHANG K, ZHANG Z, LI Z, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503.
[29] FARHAIDA MOHD ZAINON N A, RAZAK S A. Region of Interest extraction for biometric cryptosystem[C]//Proceedings of the IEEE Conference on Application, Information and Network Security, 2018: 33-37.
[30] 刘刚, 张晶, 李月龙. 基于最大内切圆算法的手掌静脉ROI提取[J]. 计算机科学, 2018, 45(8): 264-267.
LIU G, ZHANG J, LI Y L. Extraction of palm vein ROI based on maximum inscribed circle algorithm[J]. Computer Science, 2018, 45(8): 264-267.
[31] PENG M, WANG C Y, CHEN T, et al. A methodology for palm vein image enhancement and visualization[C]//Proceedings of the IEEE International Conference of Online Analysis and Computing Science, 2016: 57-60.
[32] GURUNATHAN V, BHARATHI S, SUDHAKAR R. Image enhancement techniques for palm vein images[C]//Proceedings of the International Conference on Advanced Computing and Communication Systems, 2015: 1-5.
[33] PERERA K, KHELIFI F, BELATRECHE A. A novel image enhancement method for palm vein images[C]//Proceedings of the 8th International Conference on Control, Decision and Information Technologies, 2022: 1467-1472.
[34] 娄梦莹, 袁丽莎, 刘娅琴, 等.基于自适应融合的手掌静脉增强方法[J].计算机应用, 2019, 39(4): 1176-1182.
LOU M Y, YUAN L S, LIU Y Q, et al. Palm vein enhancement method based on adaptive fusion[J].Journal of Computer Applications, 2019, 39(4): 1176-1182.
[35] HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353.
[36] JOUNG-YOUN K, LEE-SUP K, SEUNG-HO H. An advanced contrast enhancement using partially overlapped sub-block histogram equalization[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2001, 11(4): 475-484.
[37] OJALA T, PIETIKAINEN M, HARWOOD D. Performance evaluation of texture measures with classification based on Kullback discrimination of distributions[C]//Proceedings of 12th International Conference on Pattern Recognition, 1994: 582-585.
[38] LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999: 1150-1157.
[39] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2005: 886-893.
[40] PAN Z Y, WANG J, SHEN Z W, et al. Multi-layer convolutional features concatenation with semantic feature selector for vein recognition[J]. IEEE Access, 2019, 7: 90608-90619.
[41] WANG G, SUN C, SOWMYA A. Multi-weighted co-occurrence descriptor encoding for vein recognition[J]. IEEE Transactions on Information Forensics and Security, 2020, 15: 375-390.
[42] WANG J, PAN Z, WANG G, et al. Spatial pyramid pooling of selective convolutional features for vein recognition[J]. IEEE Access, 2018, 6: 28563-28572.
[43] QIN H F, EL YACOUBI M A, LIN J H, et al. An iterative deep neural network for hand-vein verification[J]. IEEE Access, 2019, 7: 34823-34837.
[44] HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7): 1527-1554.
[45] THAPAR D, JASWAL G, NIGAM A, et al. PVSNet: palm vein authentication siamese network trained using triplet loss and adaptive hard mining by learning enforced domain specific features[C]//Proceedings of the 2019 IEEE 5th International Conference on Identity, Security, and Behavior Analysis (ISBA), 2019: 1-8.
[46] KUZU R S, MAIORANA E, CAMPISI P. Vein-based biometric verification using densely-connected convolutional autoencoder[J]. IEEE Signal Processing Letters, 2020, 27: 1869-1873.
[47] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 770-778.
[48] WU T F, LENG L, KHAN M K, et al. Palmprint-palmvein fusion recognition based on deep hashing network[J]. IEEE Access, 2021, 9: 135816-135827.
[49] JADERBERG M, SIMONYAN K, ZISSERMAN A, et al. Spatial Transformer networks[J].arXiv:1506.02025, 2015.
[50] 娄梦莹, 王天景, 刘娅琴, 等.基于侧链连接卷积神经网络的手掌静脉图像识别[J].计算机应用, 2020, 40(12): 3673-3678.
LOU M Y, WANG T J, LIU Y Q, et al. Palm vein image recognition based on side chain connected convolution neural network[J].Journal of Computer Applications, 2020, 40(12): 3673-3678.
[51] HUANG G, LIU Z, WEINBERGER K Q. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2016: 2261-2269.
[52] HTET A S M, LEE H J. Contactless palm vein recognition based on attention-gated residual U-Net and ECA-ResNet[J]. Applied Sciences, 2023, 13(11): 6363.
[53] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[J]. arXiv:1505.04597, 2015.
[54] QIN H F, EL-YACOUBI M A, LI Y T, et al. Multi-scale and multi-direction GAN for CNN-based single palm-vein identification[J].IEEE Transactions on Information Forensics and Security, 2021, 16: 2652-2666.
[55] JIA W, XIA W, ZHAO Y, et al. 2D and 3D palmprint and palm vein recognition based on neural architecture search[J]. International Journal of Automation and Computing, 2021, 18(3): 377-409.
[56] ZOPH B, LE Q V. Neural architecture search with reinforcement learning[J]. arXiv:1611.01578, 2016.
[57] OBAYYA M I, EL-GHANDOUR M, ALROWAIS F. Contactless palm vein authentication using deep learning with bayesian optimization[J].IEEE Access, 2021, 9: 1940-1957.
[58] LI Y T, RUAN S, QIN H F, et al. Transformer based defense GAN against palm-vein adversarial attacks[J]. IEEE Transactions on Information Forensics and Security, 2023, 18: 1509-1523.
[59] QIN H F, GONG C Q, LI Y, et al. Label enhancement-based multiscale Transformer for palm-vein recognition[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1-17.
[60] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[J]. arXiv:2010.11929, 2020.
[61] LI Y P, LU H M, WANG Y F, et al. ViT-Cap: a novel vision Transformer-based capsule network model for finger vein recognition[J]. Applied Sciences-Basel, 2022, 12(20): 10364.
[62] GARCIA-MARTIN R, SANCHEZ-REILLO R. Vision Transformers for vein biometric recognition[J]. IEEE Access, 2023, 11: 22060-22080.
[63] 刘卫光, 刘东, 王璐. 可变形卷积网络研究综述[J].?计算机科学与探索, 2023, 17(7): 1549-1564.
LIU W G, LIU D, WANG L. Survey of deformable convolutional networks[J].Journal of Frontiers of Computer Science and Technology, 2023, 17(7): 1549-1564.
[64] JIA W, GAO J, XIA W, et al. A performance evaluation of classic convolutional neural networks for 2d and 3d palmprint and palm vein recognition[J]. International Journal of Automation and Computing, 2021, 18(1): 18-44.
[65] TAN M, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[J]. arXiv:1905.11946, 2019.
[66] HOWARD A, SANDLER M, CHU G, et al. Searching for MobileNetV3[J]. arXiv:1905.02244, 2019.
[67] SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4510-4520.
[68] SUN S S, CONG X Y, ZHANG P, et al. Palm vein recognition based on NPE and KELM[J]. IEEE Access, 2021, 9: 71778-71783.
[69] HUANG G B, ZHOU H M, DING X J, et al. Extreme learning machine for regression and multiclass classification[J]. IEEE Transactions on Systems, Man, and Cybernetics, 2012, 42(2): 513-529.
[70] ABED M H, ALI H A, ALI D, et al. Palm vein identification based on hybrid features selection model[J].?arXiv:2007.16195, 2020.
[71] NAYAR G R, THOMAS T, EMMANUEL S. Graph based secure cancelable palm vein biometrics[J]. Journal of Information Security and Applications, 2021, 62: 102991.
[72] AHMAD F, CHENG L M, KHAN A. lightweight and privacy-preserving template generation for palm-vein-based human recognition[J]. IEEE Transactions on Information Forensics and Security, 2020, 15: 184-194.
[73] DEMANET L, YING L. Wave atoms and sparsity of oscillatory patterns[J]. Applied and Computational Harmonic Analysis, 2007, 23(3): 368-387.
[74] 聂昊, 鲁玺龙, 郭文志, 等. 多模态生物特征识别技术的研究进展[J]. 生命科学仪器, 2020, 18(5): 20-28.
NIE H, LU Y L, GUO W Z, et al. Research progress and prospects of multi-modal biometric identification technology[J]. Life Science Instrument, 2020, 18(5): 20-28.
[75] ROSS A, JAIN A K. Information fusion in biometrics[J]. Pattern Recognition Letters, 2001, 24(13): 2115-2125.
[76] CHEN P, DING B J, WANG H X, et al. Design of low-cost personal identification system that uses combined palm vein and palmprint biometric features[J]. IEEE Access, 2019, 7: 15922-15931.
[77] MINAEE S, WANG Y. Fingerprint recognition using translation invariant scattering network[C]//Proceedings of the 2015 IEEE Signal Processing in Medicine and Biology Symposium (SPMB), 2015: 1-6.
[78] LI Z Q, LIANG X, FAN D D, et al. BPFNet: a unified framework for bimodal palmprint alignment and fusion[J]. arXiv:2110.01179, 2021.
[79] WANG L, ZHANG Q, QIAN Q, et al. Multispectral palm print and palm vein acquisition platform and recognition method based on convolutional neural network[J]. The Computer Journal, 2022, 65(6): 1461-1471.
[80] HUANG Y W, MA H, WANG M Y. Multimodal finger recognition based on asymmetric networks with fused similarity[J]. IEEE Access, 2023, 11: 17497-17509.
[81] JIANG L, LIU X H, WANG H X, et al. Finger vein and inner knuckle print recognition based on multilevel feature fusion network[J]. Applied Sciences, 2022, 12(21): 11182.
[82] KUANG H L, ZHONG Z H, LIU X H, et al. Palm vein recognition using convolution neural network based on feature fusion with HOG feature[C]//Proceedings of the 5th International Conference on Smart Grid and Electrical Automation (ICSGEA), 2020: 295-299.
[83] BABALOLA F O, BITIRIM Y, TOYGAR ?. Palm vein recognition through fusion of texture-based and CNN-based methods[J]. Signal, Image and Video Processing, 2021, 15(3): 459-466.