Accurate Pupil Segmentation Based on ADEU-Net Segmentation Network

doi:10.3778/j.issn.1002-8331.2108-0018

Abstract

Abstract: The pupil detection is carried out by the current image processing method, which leads to the low robustness, low precision and the inability to accurately locate the pupil in various postures of the human eye. This paper proposes an accurate method for detecting human pupils based on deep learning. The method uses the fast human-eye semantic segmentation network based on ADEU-Net to obtain the pupil region, and analyzes it to achieve the precise segmentation of the pupil. The network innovatively proposes the dilated convolution and common convolution. The two-line parallel method enhances the local refinement ability while expanding the receptive field, and introduces the attention mechanism to fully extract the semantic features. The experimental results show that through end-to-end learning, PA is improved by 5 and 35 percentage points respectively compared with U-Net and traditional image processing algorithms. The average intersection and union ratio MIoU reaches 94%, which is significantly higher than that of U-Net and traditional image processing algorithms, whose MIoU is 90% and 57% respectively. At the same time, it ensures the high segmentation speed of 83 frame/s.

Key words: deep learning, ADEU-Net, pupil detection, semantic segmentation, attention mechanism, dilated convolution

摘要： 当前通过图像处理的方法来进行瞳孔分割，导致鲁棒性不高、分割精度低以及运算量大无法满足实时性要求的问题，为此提出一种基于深度学习的人眼瞳孔精确分割方法。该方法采用基于ADEU-Net的快速人眼语义分割网络来获取瞳孔区域，实现对瞳孔的精准分割；该网络创新性地提出膨胀卷积与普通卷积双线并行的方式，在扩大感受野的同时可提升局部精细化能力，并且引入了注意力机制，以充分提取语义特征。实验结果表明，该瞳孔分割方法通过端到端学习，PA相对于U-Net、传统图像处理算法分别提高了5、35个百分点；均交并比MIoU达到94%，明显高于U-Net、传统图像处理算法90%和57%，同时保证了83?frame/s的高分割速度。

关键词: 深度学习, ADEU-Net, 瞳孔分割, 语义分割, 注意力机制, 膨胀卷积

ZHANG Hetong, YAO Kang, PEI Ronghao, DING Shangshang, FU Weiwei. Accurate Pupil Segmentation Based on ADEU-Net Segmentation Network[J]. Computer Engineering and Applications, 2023, 59(2): 212-221.

张贺童, 姚康, 裴融浩, 丁上上, 付威威. 基于ADEU-Net分割网络的瞳孔精确分割方法[J]. 计算机工程与应用, 2023, 59(2): 212-221.

References

[1] 余龙华，王宏，钟洪声.人眼检测及瞳孔定位[J].计算机工程与应用，2013，49（3）：186-189.
YU Longhua，WANG Hong，ZHONG Hongsheng.Human eye detection and pupil localization[J].Computer Engineering and Applications，2013，49（3）：186-189.
[2] 郝明刚，董秀成，黄亚勤.一种精确的人眼瞳孔定位算法[J].计算机工程，2012，38（8）：141-143.
HAO Minggang，DONG Xiucheng，HUANG Yaqin.Precise human eye pupil localization algorithm[J].Computer Engineering，2012，38（8）：141-143.
[3] MORIMOTO C H，KOONS D，AMIR A，et al.Pupil detection and tracking using multiple light sources[J].Image and Vision Computing，2000，18（4）：331-335.
[4] 黄何，牛娇红，胡建国.基于网络摄像头的实时瞳孔追踪系统设计[J].信息化研究，2014（2）：17-22.
HUANG He，NIU Jiaohong，HU Jianguo.A real-time pupil tracking system based on web-camera[J].Informatization Research，2014（2）：17-22.
[5] ZHU Y，CHEN W，ZHAN X，et al.Head mounted pupil tracking using convolutional neural network[J].arXiv：1805.00311，
2018.
[6] BASTOS C A C M，TSANG I R，VASCONCELOS G S，et al.Pupil segmentation using Pulling & Pushing and BSOM neural network[C]//IEEE International Conference on Systems，2012.
[7] RONNEBERGER O，FISCHER P，BROX T.U-Net：convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention，2015.
[8] SEVASTOPOLSKY A.Optic disc and cup segmentation methods for glaucoma detection with modification of U-Net convolutional neural network[J].Pattern Recognition and Image Analysis，2017，27（3）：618-624.
[9] LONG J，SHELHAMER E，DARRELL T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2014，39（4）：640-651.
[10] VON EICKEN T，BASU A，BUCH V，et al.U-Net：a user-level network interface for parallel and distributed computing[C]//Fifteenth ACM Symposium on Operating Systems Principles，1995.
[11] ?I?EK ?，ABDULKADIR A，LIENKAMP S S，et al.3D U-Net：learning dense volumetric segmentation from sparse annotation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2016.
[12] LU Y，LIN J，CHEN S，et al.Automatic tumor segmentation by means of deep convolutional U-Net with pre-trained encoder in PET images[J].IEEE Access，2020，8：113636-113648.
[13] HAN Y，YE J C.Framing U-Net via deep convolutional framelets：application to sparse-view CT[J].IEEE Transactions on Medical Imaging，2018，37（6）：1418-1429.
[14] OPPENHEIMER D，WELSH M.User customization of virtual network interfaces with U-Net/SLE[R].University of California at Berkeley，1998.
[15] ZEINELDIN R A，KARAR M E，COBURGER J，et al.DeepSeg：deep neural network framework for automatic brain tumor segmentation using magnetic resonance FLAIR images[J].International Journal of Computer Assisted Radiology and Surgery，2020（15）：909-920.
[16] ALOM M Z，YAKOPCIC C，TAHA T M，et al.Nuclei segmentation with recurrent residual convolutional neural networks based U-Net（R2U-Net）[C]//NAECON 2018-IEEE National Aerospace and Electronics Conference，2018.
[17] KRISHNA D S，RENUKANAND P K，SREEDHAR B K，et al.DRUNET：a dilated-residual U-Net deep learning network to segment optic nerve head tissues in optical coherence tomography images[J].Biomedical Optics Express，2018，9（7）：3244-3265.
[18] FABIJA?SKA A.Segmentation of corneal endothelium images using a U-Net-based convolutional neural network[J].Artificial Intelligence in Medicine，2018，88：1-13.
[19] KLOSS A，KAPPLER D，LENSCH H，et al.Learning where to search using visual attention[C]//IEEE/RSJ International Conference on Intelligent Robots & Systems，2016.
[20] HAO D，YANG G，LIU F，et al.Automatic brain tumor detection and segmentation using U-Net based fully convolutional networks[C]//Annual Conference on Medical Image Understanding and Analysis，2017.
[21] ZHANG C，XING Y.CT artifact reduction via U-net CNN[C]//Proceedings of SPIE，2018.
[22] YU F，KOLTUN V.Multi-scale context aggregationby dilated convolutions[C]//International Conference on Learning Representations，2016.
[23] GARCIA-GARCIA A，ORTS-ESCOLANO S，OPREA S，et al.A survey on deep learning techniques for image and video semantic segmentation[J].Applied Soft Computing，2018，70：41-65.
[24] SRIVASTAVA N，HINTON G，KRIZHEVSKY A，et al.Dropout：a simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research，2014，15（1）：1929-1958.
[25] PRECHELT L.Early stopping-but when?[M]//Neural networks tricks of the trade.Berlin，Heidelberg：Springer-Verlag，2012.
[26] 宋辉，陈浩杰，张立华.基于灰度直方图最小跨度阈值法的瞳孔分割[J].中国印刷与包装研究，2011，3（2）：29-32.
SONG Hui，CHEN Haojie，ZHANG Lihua.Pupil segmentation based on the min-cross threshold method on histograms[J].China Printing and Packaging Study，2011，3（2）：29-32.
[27] 黄丽丽，杨帆，王东强，等.基于改进型最大类间方差法的瞳孔定位方法[J].计算机工程与应用，2013，49（23）：137-140.
HUANG Lili，YANG Fan，WANG Dongqiang，et al.Pupil location method based on improved OTSU algorithm[J].Computer Engineering and Applicationss，2013，49（23）：137-140.
[28] 顾可可，付威威，董月芳，等.基于改进轮廓跟踪的瞳孔快速检测[J].红外技术，2017（6）：574-578.
GU Keke，FU Weiwei，DONG Yuefang，et al.Rapid pupil detection based on improved contour tracking[J].Infrared Technology，2017（6）：574-578.

[29] 毛顺兵.Hough变换和轮廓匹配相结合的瞳孔精确检测算法[J].计算机应用，2016，36（5）：1415-1420.

MAO Shunbing.Exact pupil detection algorithm combining Hough transformation and contour matching[J].Journal of Computer Applications，2016，36（5）：1415-1420.