Face Sketch Recognition Method Based on Cross-Batch Memory Pre-Training

doi:10.3778/j.issn.1002-8331.2109-0039

Abstract

Abstract: Face sketch recognition technology is widely used in the field of criminal investigation. It helps to narrow the scope of suspect search. Due to the insufficient number of sketched face samples, the classic deep learning model cannot achieve the ideal recognition accuracy. In response to this problem, this paper proposes a face sketch recognition method based on cross-batch pre-training. The problem of scarcity of training samples is alleviated by performing cross-batch pre-training on a limited sketch face data set. Thereby it improves the generalization ability of the face recognition model. This method relieves the GPU storage limitation through the cross-batch storage mechanism and expands the number of pre-training samples in a single batch, so as to obtain better initial model parameters. On this basis, the model is further optimized according to the triple loss to improve the network performance. The proposed method has a Rank-1 recognition accuracy of 72.53% on the UoM-SGFS sketch face dataset, and a Rank-10 recognition accuracy of 62.47% on the PRIP-VSGC dataset. Compared with CDAN, DANN, SSD and other methods, the recognition rate has been significantly improved.

Key words: cross-batch storage, face sketch recognition, pre-training, sample scarcity, batch size

摘要： 素描人脸识别技术在刑侦领域应用广泛，有助于缩小嫌疑人的搜寻范围。由于素描人脸样本数量不足，导致经典的深度学习模型无法达到理想的识别精度。针对此问题，提出一种基于跨批次预训练的素描人脸识别方法，通过在有限素描人脸数据集进行跨批次预训练的方式缓解训练样本稀缺问题，从而提高人脸识别模型的泛化能力。该方法通过跨批次存储机制缓解GPU存储限制扩大单批次预训练样本数量，从而获得更优的模型初始参数，并在其基础上根据三元组损失进一步优化模型，以提升网络性能。提出的方法在UoM-SGFS素描人脸数据集上的Rank-1识别精度为72.53%，在PRIP-VSGC数据集上Rank-10识别精度为62.47%。相比CDAN、DANN、SSD等方法识别率有显著提高。

关键词: 跨批次存储, 素描人脸识别, 预训练, 样本稀缺, 批次大小

SHAO Yuying, CAO Lin, KANG Jun, SONG Peiran, DU Kangning, GUO Yanan. Face Sketch Recognition Method Based on Cross-Batch Memory Pre-Training[J]. Computer Engineering and Applications, 2023, 59(3): 175-183.

邵玉颖, 曹林, 康峻, 宋沛然, 杜康宁, 郭亚男. 基于跨批次存储预训练的素描人脸识别方法[J]. 计算机工程与应用, 2023, 59(3): 175-183.

References

[1] JR R G U，LOBO N D V，KWON Y H.Recognizing a facial image from a police sketch[C]//Proceedings of the Second IEEE Workshop on Applications of Computer Vision，1994：129-137.
[2] PENG C，WANG N，GAO X，et al.Face recognition from multiple stylistic sketches：scenarios，datasets，and evaluation[J].Pattern Recognition，2018，84：262-272.
[3] ANZALONE L，BARRA P，BARRA S，et al.Transfer learning for facial attributes prediction and clustering[C]//International Conference on Smart City and Informatization，2019：105-117.
[4] BARRA P，BISOGNI C，NAPPI M，et al.Gender classification on 2D human skeleton[C]//2019 3rd International Conference on Bio-engineering for Smart Technologies（BioSMART），2019：1-4.
[5] CHEN J，CHEN J，WANG Z，et al.Identity-aware face super-resolution for low-resolution face recognition[J].IEEE Signal Processing Letters，2020，27：645-649.
[6] CAO D，ZHU X，HUANG X，et al.Domain balancing：face recognition on long-tailed domains[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2020：5671-5679.
[7] MENDEZ-VAZQUEZ H，BECERRA-RIERA F，MORALES-GONZALEZ A，et al.Local deep features for composite face sketch recognition[C]//2019 7th International Workshop on Biometrics and Forensics（IWBF），2019：1-6.
[8] WAN W，LEE H J.Generative adversarial multi-task learning for face sketch synthesis and recognition[C]//2019 IEEE International Conference on Image Processing（ICIP），2019：4065-4069.
[9] YANG F，WU Y，WANG Z，et al.Instance-level heterogeneous domain adaptation for limited-labeled sketch-to-photo retrieval[J].IEEE Transactions on Multimedia，2021，23：2347-2360.
[10] FENG Y，WU F，HUANG Q，et al.Cross-modality multi-task deep metric learning for sketch face recognition[C]//2019 Chinese Automation Congress（CAC），2019：2277-2281.
[11] YU S，HAN H，SHAN S，et al.Improving face sketch recognition via adversarial sketch-photo transformation[C]//2019 14th IEEE International Conference on Automatic Face & Gesture Recognition（FG 2019），2019：1-8.
[12] CHOI S，LEE S，KIM Y，et al.Hi-CMD：hierarchical cross-modality disentanglement for visible-infrared person re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2020：10257-10266.
[13] WANG Z，WANG Z X，ZHENG Y，et al.Beyond intra-modality：a survey of heterogeneous person re-identification[C]//Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence，2020：4973-4980.
[14] GALEA C，FARRUGIA R A.Matching software-generated sketches to face photos with a very deep CNN，morphed faces，and transfer learning[J].IEEE Transactions on Information Forensics and Security，2018，13（6）：1421-1431.
[15] TANG X，WANG X.Face sketch recognition[J].IEEE Transactions on Circuits & Systems for Video Technology，2004，14（1）：50-57.
[16] GALEA C，FARRUGIA R A.Fusion of intra-and inter-modality algorithms for face-sketch recognition[C]//16th International Conference on Computer Analysis of Images and Patterns，2015.
[17] KLUM S J，HAN H，KLARE B F，et al.The FaceSketchID system：matching facial composites to mugshots[J].IEEE Transactions on Information Forensics and Security，2014，9（12）：2248-2263.
[18] GALEA C，FARRUGIA R A.Face photo-sketch recognition using local and global texture descriptors[C]//24th European Signal Processing Conference，2016：2240-2244.
[19] WANG X，ZHANG H，HUANG W，et al.Cross-batch memory for embedding learning[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2020：6387-6396.
[20] BOTTOU L，FRANK E，NOCEDAL J.Optimization methods for large-scale machine[J].Society for Industrial and Applied Mathematics，2018，60（2）：223-311.
[21] LEE J D，SIMCHOWITZ M，JORDAN M I，et al.Gradient descent converges to minimizers[J].arXiv：1602.04915，2016.
[22] GE R，HUANG F，JIN C，et al.Escaping from saddle points—online stochastic gradient for tensor decomposition[J].arXiv：1503.02101，2015.
[23] HARDT M，RECHT B，SINGER Y.Train faster，generalize better：stability of stochastic gradient descent[J].arXiv：1509.01240，2015.
[24] HAN H，KLARE B F，BONNEN K，et al.Matching composite sketches to face photos：a component-based approach[J].Transactions on Information Forensics and Security，2013，8（1）：191-204.
[25] GALEA C，FARRUGIA R A.A large-scale software-generated face composite sketch database[C]//Proceedings of the 15th International Conference of the Biometrics Special Interest Group（BIOSIG），2016：1-5.
[26] RALLINGS C，THRASHER M，GUNTER C，et al.The FERET database and evaluation procedure for face-recognition algorithms[J].Image and Vision Computing，1998，16（5）：295-306.
[27] KINGMA D，BA J.Adam：a method for stochastic optimization[J].arXiv：1412.69810，2014.
[28] GANIN Y，USTINOVA E，AJAKAN H，et al.Domain-adversarial training of neural networks[J].The Journal of Machine Learning Research，2016，17（59）：1-35.
[29] LONG M，CAO Z，WANG J，et al.Conditional adversarial domain adaptation[J].arXiv：1705.10667，2017.
[30] CHEN X，WANG S，LONG M，et al.Transferability vs.discriminability：batch spectral penalization for adversarial domain adaption[C]//Proceedings of the 36th International Conference on Machine Learning，2019：1081-1090.
[31] CHERAGHI H，LEE H J.SP-net：a novel framework to identify composite sketch[J].IEEE Access，2019，7：131749-131757.
[32] WAN W，GAO Y，LEE H J.Transfer deep feature learning for face sketch recognition[J].Neural Computing and Applications，2019，31（12）：9175-9184.
[33] MITTAL P，JAIN A，GOSWAMI G，et al.Recognizing composite sketches with digital face images via SSD dictionary[C]//International Joint Conference on Biometrics，2014：1-6.
[34] MITTAL P，JAIN A，GOSWAMI G，et al.Composite sketch recognition using saliency and attribute feedback[J].Information Fusion，2017，33：86-99.
[35] MITTAL P，VATSA M，SINGH R.Composite sketch recognition via deep network—a transfer learning approach[C]//2015 International Conference on Biometrics，2015：251-256.