Kinship Verification Method of Facial Local Feature Enhancement

doi:10.3778/j.issn.1002-8331.2102-0330

Abstract

Abstract:

Kinship verification is an important branch of face recognition. The important applications of kinship verification include occasions such as search for missing children and relatives, family graph construction, social media analysis, and so on. Generally, there is a huge difference between facial images of parent and child. Therefore, how to extract discriminative features from facial images is the key to improve accuracy of kinship verification. In this paper, it proposes a method based on deep learning network to extract information of local facial regions by constructing a Local Facial Feature Enhancement Verification Net （LFFEV Net）, which can obtain the face feature representation with strong discrimination for kinship verification. LFFEV Net consists of local feature attention network and residual verification network. The local feature attention network extracts the feature map of the key local facial area, and then inputs the obtained feature map and the original image together into the residual verification network to obtain the facial features with better discrimination. Finally, the features are fused and the Family ID information is combined to verify the kinship. Extensive experiments are conducted on the publicly available kinship dataset KinFaceW-I and KinFaceW-II. The experimental results demonstrate that the method can achieve high accuracy.

Key words: deep learning, kinship verification, local feature enhancement, attention network

摘要：

亲属关系验证是人脸识别的一个重要分支，可以用于寻找失散亲人、搜寻走失儿童、构建家庭图谱、社交媒体分析等重要场景。父母和孩子的人脸图像之间往往存在较大的差异，如何从人脸中提取到有鉴别力的特征是提高亲属关系验证准确率的关键。因此，提出了一种基于深度学习和人脸局部特征增强的亲属关系验证方法，构建了人脸局部特征增强验证网络（Local Facial Feature Enhancement Verification Net，LFFEV Net），获取用于亲属关系验证的具有强鉴别力的人脸特征表示。LFFEV Net由局部特征注意力网络和残差验证网络两部分组成。局部特征注意力网络提取人脸局部关键特征，将获取的局部关键特征和对应的原始图像一同输入到残差验证网络中获取更具鉴别力的人脸特征，将特征经过融合并结合Family ID信息进行亲属关系验证。算法在公开的亲属关系数据集KinFaceW-I和KinFaceW-II上进行测试，实验结果表明，所设计的方法在亲属关系验证任务中有较高的识别率。

关键词: 深度学习, 亲属关系验证, 局部特征增强, 注意力网络

ZHENG Liang, CHEN Ya, CHEN Xiaopan, ZHENG Fengbin. Kinship Verification Method of Facial Local Feature Enhancement[J]. Computer Engineering and Applications, 2021, 57(23): 226-233.

郑亮，陈亚，陈小潘，郑逢斌. 人脸局部特征增强的亲属关系验证方法[J]. 计算机工程与应用, 2021, 57(23): 226-233.

References

[1] BRUCE V，YOUNG A.Understanding face recognition[J].British Journal of Psychology，1986，77（3）：305-327.
[2] MARTELLO M，MALONEY L.Where are kin recognition signals in the human face?[J].Journal of Vision，2004，6（12）：1356-1366.
[3] DEBRUINE L M，SMITH F G，JONES B C，et al.Kin recognition signals in adult faces[J].Vision Research，2009，49（1）：38-43.
[4] 郑远攀，李广阳，李晔.深度学习在图像识别中的应用研究综述[J].计算机工程与应用，2019，55（12）：20-36.
ZHENG Y P，LI G Y，LI Y.Survey of application of deep learning in image recognition[J].Computer Engineering and Applications，2019，55（12）：20-36.
[5] GUO J，ZHU X，ZHAO C，et al.Learning meta face recognition in unseen domains[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：6163-6172.
[6] COOTES T F，TAYLOR C J，COOPER D H，et al.Active shape models-their training and application[J].Computer Vision & Image Understanding，1995，61（1）：38-59.
[7] CRISTINACCE D，COOTES T F.Boosted regression active shape models[C]//Proceedings of the British Machine Vision Conference 2007，University of Warwick，UK，2007.
[8] CAO X，WEI Y，WEN F，et al.Face alignment by explicit shape regression[J].International Journal of Computer Vision，2014，107（2）：177-190.
[9] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[10] FANG R，TANG K D，SNAVELY N，et al.Towards computational models of kinship verification[C]//Proceedings of the International Conference on Image Processing，Hong Kong，China，2010.
[11] GUO G，WANG X.Kinship measurement on salient facial features[J].IEEE Transactions on Instrumentation and Measurement，2012，61（8）：2322-2325.
[12] LU J，ZHOU X，TAN Y P，et al.Neighborhood repulsed metric learning for kinship verification[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，36（2）：331-345.
[13] YAN H，LU J，DENG W，et al.Discriminative multimetric learning for kinship verification[J].IEEE Transactions on Information Forensics and Security，2014，9（7）：1169-1178.
[14] LU J，HU J，TAN Y.Discriminative deep metric learning for face and kinship verification[J].IEEE Transactions on Image Processing，2017，26（9）：4269-4282.
[15] ZHOU X，LU J，HU J，et al.Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments[C]//Proceedings of the 20th ACM International Conference on Multimedia，2012：725-728.
[16] SOMANATH G，KAMBHAMETTU C.Can faces verify blood-relations?[C]//2012 IEEE Fifth International Conference on Biometrics：Theory，Applications and Systems （BTAS），2012：105-112.
[17] ZHOU X，HU J，LU J，et al.Kinship verification from facial images under uncontrolled conditions[C]//Proceedings of the 19th ACM International Conference on Multimedia，2011：953-956.
[18] XIA S，SHAO M，LUO J，et al.Understanding kin relationships in a photo[J].IEEE Transactions on Multimedia，2012，14（4）：1046-1056.
[19] ZHOU X，LU J，HU J，et al.Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments[C]//Proceedings of the 20th ACM International Conference on Multimedia，2012：725-728.
[20] LU J，ZHOU X，TAN Y P，et al.Neighborhood repulsed metric learning for kinship verification[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，36（2）：331-345.
[21] ZHOU X，YAN H，SHANG Y.Kinship verification from facial images by scalable similarity fusion[J].Neurocomputing，2016，197：136-142.
[22] LU J，LIONG V E，ZHOU X，et al.Learning compact binary face descriptor for face recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（10）：2041-2056.
[23] ZHANG K，HUANG Y，SONG C，et al.Kinship verification with deep convolutional neural networks[C]//British Machine Vision Conference（BMVC），2015：1-12.
[24] ZHOU X，JIN K，XU M，et al.Learning deep compact similarity metric for kinship verification from face images[J].Information Fusion，2019，48：84-94.
[25] NANDY A，MONDAL S S.Kinship verification using deep siamese convolutional neural network[C]//2019 14th IEEE International Conference on Automatic Face & Gesture Recognition（FG 2019），2019.
[26] WOO S，PARK J，LEE J Y，et al.CBAM：convolutional block attention module[C]//European Conference on Computer Vision，2018.
[27] MNIH V，HEESS N，GRAVES A，et al.Recurrent models of visual attention[C]//Advances in Neural Information Processing Systems，2014：2204-2212.
[28] WANG F，JIANG M，QIAN C，et al.Residual attention network for image classification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3156-3164.
[29] ZHANG K，ZHANG Z，LI Z，et al.Joint face detection and alignment using multitask cascaded convolutional networks[J].IEEE Signal Processing Letters，2016，23（10）：1499-1503.
[30] HAHNLOSER R H R，SEUNG H S，SLOTINE J J.Permitted and forbidden sets in symmetric threshold-linear networks[J].Neural Computation，2003，15（3）：621.
[31] HINTON G E，SRIVASTAVA N，KRIZHEVSKY A，et al.Improving neural networks by preventing co-adaptation of feature detectors[J].arXiv：1207.0580，2012.
[32] WEN Y，ZHANG K，LI Z，et al.A discriminative feature learning approach for deep face recognition[M].[S.l.].Springer International Publishing，2016.
[33] 秦晓倩，谭晓阳，陈松灿.基于大间隔双线性分类器进行亲子关系验证[J].计算机科学与探索，2016，10（4）：524-532.
QIN X Q，TAN X Y，CHEN S C.Kinship verification through large margin bilinear classifier[J].Journal of Frontiers of Computer Science and Technology，2016，10（4）：524-532.
[34] YAN H J.Discriminative multimetric learning for kinship verification[J].Information Forensics and Security，2014，9（7）：1169-1178.
[35] YAN H，LU J，ZHOU X.Prototype-based discriminative feature learning for kinship verification[J].IEEE Transactions on Cybernetics，2017，45（11）：2535-2545.
[36] 胡正平，郭增洁，王蒙，等.基于深度卷积神经网络End-to-End模型的亲属关系认证算法[J].模式识别与人工智能，2018，31（6）：554-561.
HU Z P，GUO Z J，WANG M，et al.Kinship verification based on deep convolutional neural network end-to-end model[J].Pattern Recognition and Artificial Intelligence，2018，31（6）：554-561.