Face Anti-Spoofing Using Dual-Stream Multi-Level Fusion and Reflection Images

doi:10.3778/j.issn.1002-8331.2204-0035

Abstract

Abstract: The features extracted from RGB images are sensitive to illumination and lead to poor model generalization performance of face anti-spoofing algorithms. To deal with the problem, a face anti-spoofing method using dual-stream multi-level fusion and reflection images is proposed. Firstly, the RGB images are enhanced through multi-scale retinex with color restoration（MSRCR）, and the low, medium and high-level deep features of the RGB and resulting reflected images are extracted. Then, an effective fusion of different features at different levels is achieved through the dual multi-level feature fusion（DMFF） module. Finally, the network model is trained by joint pixel-wise binary supervision and binary supervision based binary label. Extensive experiments on five benchmark datasets, including CASIA-FASD, Replay-Attack, MSU-MFSD, OULU-NPU and SiW, show the better robustness and generalization capability of the proposed method which can fuse the multi-level features of RGB and MSRCR images and extract more intrinsic features of faces in the complex environments.

Key words: face anti-spoofing, illumination robustness, multi-level features, feature fusion

摘要： 针对从RGB图像提取到的特征对光照敏感，导致人脸活体检测模型泛化性能较差的问题，提出一种结合反射图像的双流多层次融合检测（face anti-spoofing using dual-stream multi-level fusion and reflection images，DMF-RI）算法。对RGB图像进行带色彩恢复的多尺度Retinex增强，获得反射图，并分别提取原图和反射图的低、中、高多层次深度特征；通过双流多层次特征融合模块（dual multi-level feature fusion，DMFF）实现不同层次不同特征的有效融合；联合基于二值掩码的像素级监督和基于二值标签的二分类监督训练网络模型。在CASIA-FASD、Replay-Attack、MSU-MFSD、OULU-NPU和SiW这5个人脸活体检测数据集上的多组实验表明，所提算法通过多层次融合RGB图像和MSRCR图像的深度特征，能提取人脸中更为本质的特征信息，在复杂背景条件下表现出较强的鲁棒性和泛化能力。

关键词: 人脸活体检测, 光照鲁棒性, 多层次特征, 特征融合

WANG Xin, HUANG Rui. Face Anti-Spoofing Using Dual-Stream Multi-Level Fusion and Reflection Images[J]. Computer Engineering and Applications, 2023, 59(13): 238-246.

王鑫, 黄睿. 结合反射图像的双流多层次融合人脸活体检测[J]. 计算机工程与应用, 2023, 59(13): 238-246.

References

[1] MÄÄTTÄ J，HADID A，PIETIK?INEN M.Face spoofing detection from single images using micro-texture analysis[C]//Proceedings of 2011 International Joint Conference on Biometrics（IJCB），2011：1-7.
[2] BOULKENAFET Z，KOMULAINEN J，HADID A.Face anti-spoofing based on color texture analysis[C]//Proceedings of 2015 IEEE International Conference on Image Processing（ICIP），2015：2636-2640.
[3] PATEL K，HAN H，JAIN A K.Secure face unlock：spoof detection on smartphones[J].IEEE Transactions on Information Forensics and Security，2016，11（10）：2268-2283.
[4] BOULKENAFET Z，KOMULAINEN J，HADID A.Face antispoofing using speeded-up robust features and fisher vector encoding[J].IEEE Signal Processing Letters，2016，24（2）：141-145.
[5] KOMULAINEN J，HADID A，PIETIK?INEN M.Context based face anti-spoofing[C]//Proceedings of 2013 IEEE Sixth International Conference on Biometrics：Theory，Applications and Systems（BTAS），2013：1-8.
[6] LI J，WANG Y，TAN T，et al.Live face detection based on the analysis of fourier spectra[C]//Proceedings of SPIE，2004：296-303.
[7] GALBALLY J，MARCEL S.Face anti-spoofing based on general image quality assessment[C]//Proceedings of the 22nd International Conference on Pattern Recognition，2014：1173-1178.
[8] PAN G，SUN L，WU Z，et al.Eyeblink-based anti-spoofing in face recognition from a generic webcamera[C]//Proceedings of the IEEE 11th International Conference on Computer Vision，2007：1-8.
[9] YANG J，LEI Z，LI S Z.Learn convolutional neural network for face anti-spoofing[J].arXiv：1408.5601，2014.
[10] LI L，FENG X，BOULKENAFET Z，et al.An original face anti-spoofing approach using partial convolutional neural network[C]//Proceedings of the Sixth International Conference on Image Processing Theory，Tools and Applications（IPTA），2016：1-6.
[11] 王宏飞.程鑫，赵祥模，等.光流与纹理特征融合的人脸活体检测算法[J].计算机工程与应用，2022，58（6）：170-176.
WANG H F，CHENG X，ZHAO X M，et al.Face liveness detection based on fusional optical flow and texture features[J].Computer Engineering and Applications，2022，58（6）：170-176.
[12] LIU Y，JOURABLOO A，LIU X.Learning deep models for face anti-spoofing：binary or auxiliary supervision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：389-398.
[13] KIM T，KIM Y H，KIM I，et al.BASN：enriching feature representation using bipartite auxiliary supervisions for face an-ti-spoofing[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops，2019.
[14] YU Z，ZHAO C，WANG Z，et al.Searching central difference convolutional networks for face anti-spoofing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：5295-5305.
[15] LI Z，LI H，LAM K Y，et al.Unseen face presentation attack detection with hypersphere loss[C]//Proceedings of the 2020 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP），2020：2852-2856.
[16] LIU Y，STEHOUWER J，JOURABLOO A，et al.Deep tree learning for zero-shot face anti-spoofing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：4680-4689.
[17] SAHA S，XU W，KANAKIS M，et al.Domain agnostic feature learning for image and video based face an-ti-spoofing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops，2020：802-803.
[18] WANG J，ZHANG J，BIAN Y，et al.Self-domain adaptation for face anti-spoofing[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2021：2746-2754.
[19] LAND E H，MCCANN J J.Lightness and retinex theory[J].JOSA，1971，61（1）：1-11.
[20] Choi D H，Jang I H，Kim M H，et al.Color image enhancement based on single-scale retinex with a JND-based nonlinear filter[C]//Proceedings of the IEEE International Symposium on Circuits and Systems，2007：3948-3951.
[21] JOBSON D J，RAHMAN Z，WOODELL G A.Properties and performance of a center/surround retinex[J].IEEE Transactions on Image Processing，1997，6（3）：451-462.
[22] JOBSON D J，RAHMAN Z，WOODELL G A.A multiscale retinex for bridging the gap between color images and the human observation of scenes[J].IEEE Transactions on Image Processing，1997，6（7）：965-976.
[23] GEORGE A，MARCEL S.Deep pixel-wise binary supervision for face presentation attack detection[C]//Proceedings of the International Conference on Biometrics（ICB），2019：1-8.
[24] ZHANG K，ZHANG Z，LI Z，et al.Joint face detection and alignment using multitask cascaded convolutional networks[J].IEEE Signal Processing Letters，2016，23（10）：1499-1503.
[25] ZHANG Z，YAN J，LIU S，et al.A face antispoofing database with diverse attacks[C]//Proceedings of the 5th IAPR International Conference on Biometrics（ICB），2012：26-31.
[26] CHINGOVSKA I，ANJOS A，MARCEL S.On the effectiveness of local binary patterns in face anti-spoofing[C]//Proceedings of the International Conference of Biometrics Special Interest Group（BIOSIG），2012：1-7.
[27] WEN D，HAN H，JAIN A K.Face spoof detection with image distortion analysis[J].IEEE Transactions on Information Forensics and Security，2015，10（4）：746-761.
[28] BOULKENAFET Z，KOMULAINEN J，LI L，et al.OULU-NPU：a mobile face presentation attack database with real-world variations[C]//Proceedings of the 12th IEEE International Conference on Automatic Face & Gesture Recognition，2017：612-618.
[29] CAI R，LI Z，WAN R，et al.Learning meta pattern for face anti-spoofing[J].IEEE Transactions on Information Forensics and Security，2022，17：1201-1213.
[30] FREITAS PEREIRA T，ANJOS A，MARTINO J M D，et al.LBP-TOP based countermeasure against face spoofing attacks[C]//Proceedings of the Asian Conference on Computer Vision.Berlin，Heidelberg：Springer，2012：121-132.
[31] ASIM M，MING Z，JAVED M Y.CNN based spatio-temporal feature extraction for face anti-spoofing[C]//Proceedings of the 2nd International Conference on Image，Vision and Computing（ICIVC），2017：234-238.
[32] WANG Y，NIAN F，LI T，et al.Robust face anti-spoofing with depth information[J].Journal of Visual Communication and Image Representation，2017，49：332-337.
[33] XU Z，LI S，DENG W.Learning temporal features using LSTM-CNN architecture for face anti-spoofing[C]//Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition（ACPR），2015：141-145.
[34] CHEN H，HU G，LEI Z，et al.Attention-based two-stream convolutional networks for face spoofing detection[J].IEEE Transactions on Information Forensics and Security，2019，15：578-593.
[35] ATOUM Y，LIU Y，JOURABLOO A，et al.Face anti-spoofing using patch and depth-based CNNs[C]//Proceedings of the IEEE International Joint Conference on Biometrics（IJCB），2017：319-328.
[36] YANG X，LUO W，BAO L，et al.Face anti-spoofing：Model matters，so does data[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：3507-3516.
[37] CAI R，LI H，WANG S，et al.DRL-FAS：a novel framework based on deep reinforcement learning for face anti-spoofing[J].IEEE Transactions on Information Forensics and Security，2020，16：937-951.
[38] DE FREITAS PEREIRA T，ANJOS A，DE MARTINO J M，et al.Can face anti-spoofing countermeasures work in a real world scenario?[C]//Proceedings of the 2013 International Conference on Biometrics（ICB），2013：1-8.
[39] BOULKENAFET Z，KOMULAINEN J，HADID A.Face spoofing detection using colour texture analysis[J].IEEE Transactions on Information Forensics and Security，2016，11（8）：1818-1830.
[40] JOURABLOO A，LIU Y，LIU X.Face de-spoofing：anti-spoofing via noise modeling[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：290-306.
[41] LI H，PAN S J，WANG S，et al.Domain generalization with adversarial feature learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：5400-5409.
[42] SHAO R，LAN X，LI J，et al.Multi-adversarial discriminative deep domain generalization for face presentation attack detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：10023-10031.
[43] WANG G，HAN H，SHAN S，et al.Cross-domain face presentation attack detection via multi-domain disentangled representation learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：6678-6687.
[44] QIN Y，ZHANG W，SHI J，et al.One-class adaptation face anti-spoofing with loss function search[J].NeuroComputing，2020，417：384-395.
[45] WANG G，HAN H，SHAN S，et al.Unsupervised adversarial domain adaptation for cross-domain face presentation attack detection[J].IEEE Transactions on Information Forensics and Security，2020，16：56-69.
[46] VAN DER MAATEN L，HINTON G.Visualizing data using t-SNE[J].Journal of Machine Learning Research，2008，9：2579-2605.