Multi-Stream Threshold Shrinkage and Fusion Network for Product Surface Defect Detection

doi:10.3778/j.issn.1002-8331.2201-0405

Abstract

Abstract: The task of product surface defect detection focuses on the automatic detection and segmentation of abnormal defect areas. In practice, the detection of product surface defects remains a challenging task due to the degrading effects of noise and the complexity and variety of defect types. To cope with these problems, this paper proposes a multi-stream threshold shrinkage and fusion network for product surface defect detection. In each stream of different scales, in order to cope with noise corruption, the proposed network configures an adaptive threshold shrinkage denoising module. This module can autonomously learn the horizontal and vertical shrinkage thresholds in the dual branches, and remove the interference noise from the features while retaining the effective background information, therefore realizing adaptive denoising. In order to locate the defective object more accurately, a contextual 3D attention fusion module is designed to generate 3D attention maps by horizontal and vertical aggregation to enhance the abnormal region features. Finally, parallel multi-scale features are fused to achieve effective detection of different scales and different types of defects. This paper compares the constructed model on SD-900 and MVTec-AD datasets with the latest eight methods. The experimental results show that the model in this paper can effectively improve the detection accuracy and maintain robustness to noise interference, and the ablation experiments also verify the effectiveness of the adaptive threshold shrinkage denoising module and the contextual 3D attention fusion module.

Key words: defect detection, multi-scale fusion network, self-adaptive threshold denoising, contextual 3D attention

摘要： 产品表面缺陷检测任务的重点是对产品表面图像中的异常缺陷区域进行自动检测和分割。然而实际应用中，由于噪声的退化影响和缺陷类型的复杂多样，产品表面缺陷检测仍然是一项具有挑战性的任务。为了应对这些问题，提出了产品表面缺陷检测的多通路阈值收缩融合网络。在各尺度通路中，为了降低噪声干扰，该网络设计了自适应阈值收缩去噪模块，通过双支路自主学习水平和垂直方向的收缩阈值，去除特征中的干扰噪声并且保留有效背景信息，从而实现自适应去噪。为了更准确定位缺陷对象，设计了上下文三维注意力融合模块，通过水平聚合和垂直聚合生成三维注意力图，增强异常区域特征。最终将平行的多尺度特征融合，实现对不同尺度以及不同类型缺陷的有效检测。将所构建模型在SD-900和MVTec-AD数据集上与最新的8种方法进行比较，实验结果表明该模型能够有效提升检测精度，并能够对噪声干扰保持鲁棒性，消融实验也验证了自适应阈值收缩去噪模块和上下文三维注意力模块融合的有效性。

关键词: 缺陷检测, 多尺度融合网络, 自适应阈值去噪, 上下文三维注意力

GENG Yubiao, YUE Zhiyuan, YAN Qiming, SUN Yubao. Multi-Stream Threshold Shrinkage and Fusion Network for Product Surface Defect Detection[J]. Computer Engineering and Applications, 2023, 59(10): 162-170.

耿玉标, 岳志远, 闫麒名, 孙玉宝. 产品表面缺陷检测的多通路阈值收缩融合网络[J]. 计算机工程与应用, 2023, 59(10): 162-170.

References

[1] WEI X，YANG Z，LIU Y，et al.Railway track fastener defect detection based on image processing and deep learning techniques：a comparative study[J].Engineering Applications of Artificial Intelligence，2019，80：66-81.
[2] TOLBA A S，RAAFAT H M.Multiscale image quality measures for defect detection in thin films[J].The International Journal of Advanced Manufacturing Technology，2015，79：113-122.
[3] NG H F.Automatic thresholding for defect detection[J].Pattern Recognition Letters，2006，27（14）：1644-1649.
[4] CAPIZZI G，SCIUTO G L，NAPOLI C，et al.Automatic classification of fruit defects based on co-occurrence matrix and neural networks[C]//2015 Federated Conference on Computer Science and Information Systems，2015：861-867.
[5] BODNAROVA A，BENNAMOUN M，LATHAM S J.A constrained minimisation approach to optimise Gabor filters for detecting flaws in woven textiles[C]//2000 IEEE International Conference on Acoustics，Speech，and Signal Processing，2000，6：3606-3609.
[6] YU Z Y，WU X J，GU X D.Fully convolutional networks for surface defect inspection in industrial environment[C]//11th International Conference on Computer Vision Systems.Cham：Springer，2017：417-426.
[7] 李忠海，白秋阳，王富明，等.基于语义分割的钢轨表面缺陷实时检测系统[J].计算机工程与应用，2021，57（12）：248-256.
LI Z H，BAI Q Y，WANG F M，et al.Real-time detection system of rail surface defects based on semantic segmentation[J].Computer Engineering and Applications，2021，57（12）：248-256.
[8] BOŽIČ J，TABERNIK D，SKO?AJ D.End-to-end training of a two-stage neural network for defect detection[C]//2020 25th International Conference on Pattern Recognition，2021：5619-5626.
[9] LIANG X，ZHANG J，ZHUO L，et al.Small object detection in unmanned aerial vehicle images using feature fusion and scaling-based single shot detector with spatial context analysis[J].IEEE Transactions on Circuits and Systems for Video Technology，2020，30（6）：1758-1770.
[10] DONG H W，SONG K C，HE Y，et al.PGA-Net：pyramid feature fusion and global context attention network for automated surface defect detection[J].IEEE Transactions on Industrial Informatics，2020，16（12）：7448-7458.
[11] CAO J，YANG G，YANG X.A pixel-level segmentation convolutional neural network based on deep feature fusion for surface defect detection[J].IEEE Transactions on Instrumentation and Measurement，2020，70：1-12.
[12] SONG G，SONG K，YAN Y.EDRNet：encoder-decoder residual network for salient object detection of strip steel surface defects[J].IEEE Transactions on Instrumentation and Measurement，2020，69（12）：9709-9719.
[13] SONG G，SONG K，YAN Y.Saliency detection for strip steel surface defects using multiple constraints and improved texture features[J].Optics and Lasers in Engineering，2020，128：106000.
[14] BERGMANN P，FAUSER M，SATTLEGGER D，et al.MVTec AD—a comprehensive real-world dataset for un-
supervised anomaly detection[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：9592-9600.
[15] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[16] ISOGAWA K，IDA T，SHIODERA T，et al.Deep shrinkage convolutional neural network for adaptive noise reduction[J].IEEE Signal Processing Letters，2017，25（2）：224-228.
[17] HOU Q，ZHOU D，FENG J.Coordinate attention for efficient mobile network design[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：13713-13722.
[18] DE BOER P T，KROESE D P，MANNOR S，et al.A tutorial on the cross-entropy method[J].Annals of Operations Research，2005，134（1）：19-67.
[19] RAHMAN M A，WANG Y.Optimizing intersection-over-union in deep neural networks for image segmentation[C]//International Symposium on Visual Computing.Cham：Springer，2016：234-244.
[20] WANG Z，SIMONCELLI E P，Bovik A C.Multiscale structural similarity for image quality assessment[C]//Proceedings of the 37th Asilomar Conference on Signals，Systems & Computers，2003，2：1398-1402.
[21] HOU Q，CHENG M M，HU X，et al.Deeply supervised salient object detection with short connections[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：3203-3212.
[22] WU Z，SU L，HUANG Q.Cascaded partial decoder for fast and accurate salient object detection[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：3907-3916.
[23] LUO Z，MISHRA A，ACHKAR A，et al.Non-local deep features for salient object detection[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：6609-6617.
[24] DENG Z，HU X，ZHU L，et al.R3Net：recurrent residual refinement network for saliency detection[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence，2018：684-690.
[25] LIU J J，HOU Q，CHENG M M，et al.A simple pooling-based design for real-time salient object detection[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：3917-3926.
[26] QIN X，ZHANG Z，HUANG C，et al.BASNet：boundary-aware salient object detection[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：7479-7489.
[27] LIU N，HAN J，YANG M H.PICANet：learning pixel-wise contextual attention for saliency detection[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：3089-3098.