Research on Fusion of Visible and Near-Infrared Iris Based on Multi-Tasking Learning

doi:10.3778/j.issn.1002-8331.2104-0053

Abstract

Abstract: Aiming at the problems in the visible light iris recognition that the iris texture features are not obvious and the reflected light spots, This paper proposes an unsupervised fusion model of visible light and near-infrared based on multi-task learning（MTIris-Fusion）. The method designs an end-to-end fusion backbone network based on improved DenseU-Net. A loss function is designed to adaptively weigh the retention of important information of the source image, and adaptively maintain the similarity between the fusion result and the source image, so as to achieve the goal of unsupervised. Subsequently, the weights of the multi-converged tasks are updated through the elastic weight consolidation mechanism to avoid catastrophic forgetting in the multi-task network. Experiments on the PolyU Cross Spectral Iris data set show that compared with other methods, this method takes into account the color texture of the visible light iris and the structure information of the near-infrared image, and effectively suppresses the spot noise in the visible light image. It has important application value in the field of iris image quality enhancement.

Key words: multi-tasking learning, iris image fusion, visible light, near-infrared, loss function

摘要： 针对在可见光虹膜识别中存在的虹膜纹理特征不明显与反射光斑问题，提出一种基于多任务学习的可见光与近红外无监督融合模型（MTIris-Fusion）。设计了基于改进DenseU-Net的端到端融合骨干网。设计了自适应权衡源图像重要信息保留度的损失函数，自适应保持融合结果与源图像之间的相似度，达到无监督的目的。通过弹性权重巩固（EWC）机制更新多融合任务的权重，避免了多任务网络中的灾难性遗忘。在PolyU Cross Spectral Iris数据集上的实验表明，与其他方法相比，该方法兼顾可见光虹膜的颜色纹理与近红外图像的结构信息并有效抑制了可见光图像中的光斑噪声，在虹膜图像质量增强领域具有重要应用价值。

关键词: 多任务学习, 虹膜图像融合, 可见光, 近红外, 损失函数

YOU Xuan’ang, ZHAO Peng, MU Xiaodong, ZHU Yongqing, SHEN Danyao. Research on Fusion of Visible and Near-Infrared Iris Based on Multi-Tasking Learning[J]. Computer Engineering and Applications, 2022, 58(21): 197-204.

尤轩昂, 赵鹏, 慕晓冬, 朱永清, 沈丹瑶. 基于多任务学习的可见光与近红外虹膜融合研究[J]. 计算机工程与应用, 2022, 58(21): 197-204.

References

[1] BURGE M J，MONACO M K.Multispectral iris fusion for enhancement，interoperabilityand cross wavelength matching[C]//Proceedings of SPIE，International Society for Optics and Photonics，2009，7334：73341D.
[2] WANG K，KUMAR A.Cross-spectral iris recognition using CNN and supervised discrete hashing[J].Pattern Recognition，2019，86：85-98.
[3] SEDDON J M，SAHAGIAN C R，GLYNN R J，et al.Evaluation of an iris color classification system[J].Investigative Ophthalmolology Visual Science，1990，31：1592-1598.
[4] HOSSEINI S M，ARAABI B N，SOLTANIAN-ZADEH H.Shape analysis of stroma for iris recognition[C]//International Conference on Biometrics.Berlin，Heidelberg：Springer，2007：790-799.
[5] FRANSSEN L，COPPENS J E，THOMAS J T P，et al.Grading of iris color with an extended photographic reference set[J].Journal Optometry，2008，1（1）：36-40.
[6] SCHLETT T，RATHGEB C，BUSCH C.Multi-spectral iris segmentation in visible wavelengths[C]//Proceedings of International Conference on Biometrics（ICB），2018：190-194.
[7] 高雪琴，刘刚，肖刚，等.基于FPDE的红外与可见光图像融合算法[J].自动化学报，2020，46（4）：796-804.
GAO X Q，LIU G，XIAO G，et al.Fusion algorithm of infrared and visible images based on FPDE[J].Acta Automatica Sinica，2020，46（4）：796-804.
[8] ZHAO Z，AJAY K.An accurate iris segmentation framework under relaxed imaging constraints using total variation model[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：3828-3836.
[9] 唐超影，浦世亮，叶鹏钊，等.基于卷积神经网络的低照度可见光与近红外图像融合[J].光学学报，2020，40（16）：37-45.
TANG C Y，PU S L，YE P Z，et al.Fusion of low-illuminance visible and near-infrared images based on convolutional neural networks[J].Acta Optica Sinica，2020，40（16）：37-45.
[10] 王娟，柯聪，刘敏，等.神经网络框架下的红外与可见光图像融合算法综述[J].激光杂志，2020，41（7）：7-12.
WANG J，KE C，LIU M，et al.Overview of infrared and visible image fusion algorithms based on neural network framework[J].Chinese Journal of Laser，2020，41（7）：7-12.
[11] 陈卓，方明，柴旭，等.红外与可见光图像融合的U-GAN模型[J].西北工业大学学报，2020，38（4）：904-912.
CHEN Z，FANG M，CHAI X，et al.U-GAN model for infrared and visible images fusion[J].Journal of Northwestern Polytechnical University，2020，38（4）：904-912.
[12] LI W，JIA L，DU J.Multi-modal sensor medical image fusion based on multiple salient features with guided image filter[J].IEEE Access，2019，7：173019-173033.
[13] JIAN L，YANG X，LIU Z，et al.SEDRFuse：a symmetric encoder-decoder with residual block network for infrared and visible image fusion[J].arXiv：1905.11447，2019.
[14] HUI L，WU X J.DenseFuse：a fusion approach to infrared and visible images[J].IEEE Transactions on Image Processing，2018，28（5）：2614-2623.
[15] 宫雅卓，张大鹏，施鹏飞，等.针对东方人黑色虹膜识别的光谱选择[J].中国科学：信息科学，2012，42（7）：859-868.
GONG Y Z，ZHANG D P，SHI P F，et al.Wavelength band selection for black-based iris recognition of east asians[J].Scientia Sinica Informatics，2012，42（7）：859-868.
[16] NALLA P R，KUMAR A.Toward more accurate iris recognition using cross-spectral matching[J].IEEE Transactions on Image Processing，2017，26（1）：208-221.
[17] RADFORD A，METZ L，CHINTALA S.Unsupervisedrepesentation learning with deep convolutional generative adversarial networks[C]//International Conference on Learning Representations，2016：780-789.
[18] XU H，MA J Y，LE Z L，et al.FusionDN：a unified densely connected network for image fusion[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：12484-12491.
[19] ROY K，SHELTON J，ESTERLINE A.A brief survey on multispectral iris recognition[J].International Journal of Applied Pattern Recognition，2016，3（4）：308-323.
[20] ZHANG X，YE P，XIAO G.VIFB：a visible and infrared image fusion benchmark[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops，2020：104-105.
[21] KIRKPATRICK J，PASCANU R，RABINOWITZ N，et al.Overcoming catastrophic forgetting in neural networks[J].Proceedings of the National Academy of Sciences，2017，114（13）：3521-3526.