边界感知SegFormer网络的阵列目标图像分割方法

doi:10.3778/j.issn.1002-8331.2308-0091

摘要/Abstract

摘要： 针对工业场景下阵列目标图像存在非均匀背景、缺陷干扰和弱边缘导致目标分割精度低的问题，提出边界感知SegFormer网络的阵列目标图像分割方法。针对固定种子易受背景和缺陷干扰的问题，提出自适应种子搜索策略。该策略利用种子位置与目标定位精度的相关性构建种子分布热力图，并在热力图的引导下自适应搜索理想种子目标，实现阵列目标的高精度全局分割。设计边界感知SegFormer网络进行局部分割，利用递归门控卷积强调特征的长距离和高阶空间交互，改进的门控残差边界细化模块能够学习更丰富的边缘信息，同时引入混合损失函数加强对区域内部和边缘像素的监督，引导网络更好地学习目标边缘特征，提高边界分割精度。在自建晶粒数据集和语义分割数据集Cityscapes上的验证实验表明，提出的分割方法能在背景不均、缺陷污染、边缘对比度低的高分辨率阵列目标图像中完整精确地分割目标，并具有较高的实时性。

关键词: 图像分割, 阵列目标, 自适应种子搜索, 边界感知, SegFormer

Abstract: Aiming at the problem of low target segmentation accuracy due to the existence of non-uniform background, defect interference and weak edges in array target images in industrial scenarios, an array target image segmentation method with boundary-aware SegFormer network is proposed. Firstly, an adaptive seed searching strategy is proposed for the problem that fixed seeds are susceptible to background and defect interference. This strategy uses the correlation between seed location and target positioning accuracy to construct a heat map of seed distribution, and adaptively searches for ideal seed targets under the guidance of the heat map to achieve high-precision global segmentation of array targets. Secondly, the boundary-aware SegFormer network is designed for local segmentation, using recursive gated convolution to emphasize long-range and higher-order spatial interactions of features, an improved gated residual boundary refinement module to learn richer edge information, and the introduction of a hybrid loss function to enhance the supervision of the region interior and the edge pixels, which guides the network to better learn the target edge features and improve the boundary segmentation accuracy. Validation experiments on the self-built wafer dataset and the semantic segmentation dataset Cityscapes show that the proposed segmentation method is able to completely and accurately segment targets in high-resolution array target images with uneven backgrounds, defective contaminations, and low edge contrast with high real-time performance.

Key words: image segmentation, array target, adaptive seed searching, boundary awareness, SegFormer

吕扬, 吴静静, 庄祉珊, 安聪颖. 边界感知SegFormer网络的阵列目标图像分割方法[J]. 计算机工程与应用, 2024, 60(24): 188-199.

LYU Yang, WU Jingjing, ZHUANG Zhishan, AN Congying. Array Target Image Segmentation with Boundary-Aware SegFormer Network[J]. Computer Engineering and Applications, 2024, 60(24): 188-199.

参考文献

[1] NG J Q, YANG Z X, YING H, et al. Bougie location algorithm of lattice biochip based on YOLOv5[J]. Journal of Electronic Imaging, 2022, 31: 063006.
[2] CHEN X, CHEN J, HAN X, et al. A light-weighted CNN model for wafer structural defect detection[J]. IEEE Access, 2020, 8: 24006-24018.
[3] XU S, CHENG Z, GAO Y, et al. Visual wafer dies counting using geometrical characteristics[J]. IET Image Processing, 2014, 8(5): 280-288.
[4] 杨东旭. 基于机器视觉的芯片分选系统研究[D]. 北京: 中国电子科技集团公司电子科学研究院, 2022.
YANG D X. Research on chip sorting system based on machine vision[D]. Beijing: Institute of Electronic Science and Technology, China Electronics Technology Group Corporation, 2022.
[5] 郭毅强. 晶圆表面缺陷视觉检测研究[D]. 深圳: 哈尔滨工业大学, 2019.
GUO Y Q. Research on visual inspection of wafer surface defects[D]. Shenzhen: Harbin Institute of Technology, 2019.
[6] CHEN H C. Automated detection and classification of defective and abnormal dies in wafer images[J]. Applied Sciences, 2020, 10(10): 3423.
[7] 于志斌, 胡泓. 基于YOLO算法与机器视觉的晶圆片表面缺陷检测研究[J]. 新型工业化, 2021, 11(12): 114-117.
YU Z B, HU H. Research on surface defect detection of wafer based on YOLO algorithm and machine vision[J]. The Journal of New Industrialization, 2021, 11(12): 114-117.
[8] LIN Y. Wafer pattern counting, detection and classification based on encoder-decoder CNN structure[C]//Proceedings of the 2022 Intermountain Engineering, Technology and Computing (IETC), Orem, UT, USA, May 13-14, 2022. Piscataway: IEEE Press, 2022: 1-5.
[9] HELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, June 07-12. New York: IEEE Press, 2015: 3431-3440.
[10] QIN X, ZHANG Z, HUANG C, et al. BASNet: boundary-aware salient object detection[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, June 15-20, 2019. New York: IEEE Press, 2019: 7471-7481.
[11] ZHOU H, XIE X, LAI J, et al. Interactive two-stream decoder for accurate and fast saliency detection[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, June 13-19, 2020. New York: IEEE Press, 2020: 9138-9147.
[12] WANG C, ZHANG Y, CUI M, et al. Active boundary loss for semantic segmentation[C]//Proceedings of the AAAI Conference on Artificial Intelligence, California, Feb 2-March 3, 2022: 2397-2405.
[13] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4): 834-848.
[14] CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the 2018 15th European Conference on Computer Vision (ECCV), Munich, Germany, September 8-14, 2018. Cham: Springer, 2018: 801-818.
[15] HOWARD A G, ZHU M, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[J]. arXiv:1704.04861, 2017.
[16] SANDLER M, HOWARD A G, ZHU M, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, June 18-22, 2018. New York: IEEE Press, 2018: 4510-4520.
[17] ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, July 21-26, 2017: 6230-6239.
[18] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[J]. arXiv:1706.05587, 2017.
[19] ZHENG S, LU J, ZHAO H, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, June 19-25, 2021. New York: IEEE Press, 2021: 6877-6886.
[20] XIE E, WANG W, YU Z, et al. SegFormer: simple and efficient design for semantic segmentation with transformers[J]. arXiv:2105.15203, 2021.
[21] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[J]. arXiv:1412.7062, 2014.
[22] YAN H, ZHANG C, WU M. Lawin transformer: improving semantic segmentation transformer with multi-scale representations via large window attention[J]. arXiv:2201.01615, 2022.
[23] DULAM R V S, FEDDERS E R, MAHONEY A R, et al. ConvSegFormer-a convolution aided SegFormer architecture for?detection of?discontinuities in?wrapped interferometric phase imagery of?sea ice[C]//Proceedings of the 22nd Scandinavian Conference on Image Analysis (SCIA 2023), Sirkka, Finland, April 18-21, 2023. Cham: Springer Nature Switzerland, 2023: 203-213.
[24] SU H, YE Y, HUA W, et al. Sasformer: transformers for sparsely annotated semantic segmentation[J]. arXiv:2212.
02019, 2022.
[25] 倪天宇. 晶圆表面缺陷自动检测技术的研究[D]. 上海: 东华大学, 2022.
NI T Y. Research on automatic detection thechnology of wafer surface defects[D]. Shanghai: Donghua University, 2022.
[26] RAO Y, ZHAO W, TANG Y, et al. Hornet: efficient high-order spatial interactions with recursive gated convolutions[J]. arXiv:2207.14284, 2022.
[27] CORDTS M, OMRAN M, RAMOS S, et al. The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, June 26-July 1, 2016. New York: IEEE Press, 2016: 3213-3223.
[28] CHENG B, GIRSHICK R B, DOLL'AR P, et al. Boundary iou: improving object-centric image segmentation evaluation[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, June 19-25, 2021. New York: IEEE Press, 2021: 15329-15337.
[29] HONG Y, PAN H, SUN W, et al. Deep dual-resolution networks for real-time and accurate semantic segmentation of road scenes[J]. arXiv:2101.06085, 2021.
[30] WANG J, GOU C X, WU Q, et al. Rtformer: efficient design for real-time semantic segmentation with transformer[J]. arXiv:2210.07124, 2022.
[31] DONG B, WANG P, WANG F. Head-free lightweight semantic segmentation with linear transformer[C]//Proceedings of the AAAI Conference on Artificial Intelligence, Washington DC, June 27, 2023. Menlo Park: AAAI Press, 2023: 516-524.