Improved DeepLabv3+ Model for Surface Defect Detection on Steel Plates

doi:10.3778/j.issn.1002-8331.2210-0249

Abstract

Abstract: To address problems of rough edge segmentation, missed detection and high false detection rate in steel surface defect detection, a multi-scale feature fusion detection method based on DeepLabv3+ with attention mechanism is proposed. Firstly, in the decoding region of the DeepLabv3+ network, the multi-scale feature information is fully utilized and the leap-frog feature fusion is optimized to retain the shallow features, while a more refined up-sampling operation on the deep features is performed to obtain finer defect edges. Secondly, a coordinate attention mechanism is introduced into the coding region backbone network ResNet101 to enhance the capability of the feature extraction and improve the segmentation accuracy. In addition, an optimized loss function combining weighted Dice loss and binary cross entropy loss（BCEloss） is designed to alleviate the problem of sample imbalance and improve segmentation accuracy. The Dice coefficient and mIoU values of the advanced DeepLabv3+ network are improved by 6.0% and 7.92% respectively, with more accurate edge segmentation of scratch defects and significant improvement in the segmentation of pits, edge cracks and iron oxide defects. The experimental results validate the effectiveness of the method in dealing with the steel surface defect problem.

Key words: surface defect detection, DeepLabv3+ network, coordinate attention mechanism, image semantic segmentation, image enhancement

摘要： 针对钢板表面缺陷检测中存在的边缘分割粗糙、漏检和误检率高等问题，提出了一种引入注意力机制的多尺度特征融合的DeepLabv3+检测方法。在DeepLabv3+网络的解码区中，充分利用多尺度特征信息，对跃层特征融合进行优化，保留浅层特征并对深层特征进行了细化的上采样操作，获得更精细的缺陷边缘；在编码区主干网络ResNet101中引入坐标注意力机制，增强特征提取能力，提高分割准确率。设计了加权Dice损失和二元交叉熵损失（BCEloss）结合的优化损失函数来缓解样本不均衡的问题，提高分割精度。改进DeepLabv3+网络的Dice系数和mIoU值分别提高了6.0%和7.92%，刮痕缺陷边缘分割更准确，对凹坑、边缘裂纹与氧化铁皮缺陷的分割效果提升明显，实验结果验证了该方法处理钢板表面缺陷问题的有效性。

关键词: 表面缺陷检测, DeepLabv3+网络, 坐标注意力机制, 图像语义分割, 图像增强

FAN Yaoyao, WANG Xingfen, LIU Yahui. Improved DeepLabv3+ Model for Surface Defect Detection on Steel Plates[J]. Computer Engineering and Applications, 2023, 59(16): 150-158.

范瑶瑶, 王兴芬, 刘亚辉. 改进DeepLabv3+网络的钢板表面缺陷检测研究[J]. 计算机工程与应用, 2023, 59(16): 150-158.

References

[1] 寇旭鹏，刘帅君，麻之润.基于Faster-RCNN的钢带缺陷检测方法[J].中国冶金，2021，31（4）：77-83.
KOU X P，LIU S J，MA Z R.Defect detection method of steel strip based on Faster-RCNN[J].China Metallurgy，2021，31（4）：77-83.
[2] 李少波，杨静，王铮，等.缺陷检测技术的发展与应用研究综述[J].自动化学报，2020，46（11）：2319-2336.
LI S B，YANG J，WANG Z，et al.Review of development and application of defect detection technology[J].Acta Automatica Sinica，2020，46（11）：2319-2336.
[3] BORSELLI A，COLLA V，VANNUCCI M，et al.A fuzzy inference system applied to defect detection in flat steel production[C]//2010 IEEE International Conference on Fuzzy Systems，2010：1-6.
[4] CHU M，LIU X，GONG R，et al.Multi-class classification method using twin support vector machines with multi-information for steel surface defects[J].Chemometrics and Intelligent Laboratory Systems，2018，176：108-118.
[5] 徐镪，朱洪锦，范洪辉，等.改进的YOLOv3网络在钢板表面缺陷检测研究[J].计算机工程与应用，2020，56（16）：265-272.
XU Q，ZHU H J，FAN H H，et al.Study on detection of steel plate surface defects by improved YOLOv3 network[J].Computer Engineering and Applications，2020，56（16）：265-272.
[6] KIM M S，PARK T，PARK P G.Classification of steel surface defect using convolutional neural network with few images[C]//2019 12th Asian Control Conference，2019：1398-1401.
[7] 张广世，葛广英，朱荣华，等.基于改进YOLOv3网络的齿轮缺陷检测[J].激光与光电子学进展，2020，57（12）：153-161.
ZHANG G S，GE G Y，ZHU R H，et al.Gear defect detection based on the improved YOLOv3 Network[J].Laser & Optoelectronics Progress，2020，57（12）：153-161.
[8] 田萱，王亮，丁琪.基于深度学习的图像语义分割方法综述[J].软件学报，2019，30（2）：440-468.
TIAN X，WANG L，DING Q.Review of image semantic segmentation based on deep learning[J].Journal of Software，2019，30（2）：440-468.
[9] 程万胜.钢板表面缺陷检测技术的研究[D].哈尔滨：哈尔滨工业大学，2008.
CHENG W S.Study on detection technology for steel strip surface defects[D].Harbin：Harbin Institute of Technology，2008.
[10] XU K，XU Y，ZHOU P，et al.Application of RNAMlet to surface defect identification of steels[J].Optics and Lasers in Engineering，2018，105：110-117.
[11] 邓勇，黄远伟，赖治屹.钢板缺陷识别的Volterra-SVM模型研究[J].机械科学与技术，2023，42（1）：132-138.
DENG Y，HUANG Y W，LAI Z Y.Study on Volterra-SVM model or defect recognition of steel plate[J].Mechanical Science and Technology for Aerospace Engineering，2023，42（1）：132-138.
[12] HASNI H，ALAVI A H，JIAO P，et al.Detection of fatigue cracking in steel bridge girders：a support vector machine approach[J].Archives of Civil and Mechanical Engineering，2017，17（3）：609-622.
[13] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.Image-
Net classification with deep convolutional neural networks[J].Association for Computing Machinery，2017，60（6）：84-90.
[14] 陶显，侯伟，徐德.基于深度学习的表面缺陷检测方法综述[J].自动化学报，2021，47（5）：1017-1034.
TAO X，HOU W，XU D.A survey of surface defect detection methods based on deep learning[J].Acta Automatica Sinica，2021，47（5）：1017-1034.
[15] WANG S，XIA X，YE L，et al.Automatic detection and classification of steel surface defect using deep convolutional neural networks[J].Metals，2021，11（3）：388.
[16] 李钧正，殷子玉，乐心怡.基于小样本学习的钢板表面缺陷检测技术[J].航空科学技术，2021，32（6）：65-70.
LI J Z，YIN Z Y，LE X Y.Surface defect detection for steel plate with small dataset[J].Aeronautical Science & Technology，2021，32（6）：65-70.
[17] HUANG Z，WU J，XIE F.Automatic surface defect segmentation for hot-rolled steel strip using depth-wise separable U-shape network[J].Materials Letters，2021，301：130271.
[18] 徐辉，祝玉华，甄彤，等.深度神经网络图像语义分割方法综述[J].计算机科学与探索，2021，15（1）：47-59.
XU H，ZHU Y H，ZHEN T，et al.Survey of image semantic segmentation methods based on deep neural network[J].Journal of Frontiers of Computer Science and Technology，2021，15（1）：47-59.
[19] 张鑫，姚庆安，赵健，等.全卷积神经网络图像语义分割方法综述[J].计算机工程与应用，2022，58（8）：45-57.
ZHANG X，YAO Q A，ZHAO J，et al.Image semantic segmentation based on fully convolutional neural network[J].Computer Engineering and Applications，2022，58（8）：45-57.
[20] LI H，X P，FAN H，et al.DFANet：deep feature aggregation for real-time semantic segmentation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：9522-9531.
[21] LI Y，ZHANG J，CHENG Y，et al.Semantics-guided multi-level RGB-D feature fusion for indoor semantic segmentation[C]//Proceedings of the 2017 IEEE International Conference on Image Processing，2017：1262-1266.
[22] LONG J，SHELHAMER E，DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition，2015：3431-3440.
[23] RONNEBERGER O，FISCHER P，BROX T.U-net：Convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2015：234-241.
[24] BADRINARAYANAN V，KENDALL A，CIPOLLA R.SegNet：a deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（12）：2481-2495.
[25] CHEN L C，ZHU Y，PAPANDREOU G，et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the 15th European Conference on Computer Vision，2018：801-818.
[26] ZHAO H，SHI J，QI X，et al.Pyramid scene parsing network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：2881-2890.
[27] KIM B，CHO S.Image-based concrete crack assessment using mask and region-based convolutional neural network[J].Structural Control and Health Monitoring，2019，26（8）：1545-2255.
[28] 高广尚.深度学习推荐模型中的注意力机制研究综述[J].计算机工程与应用，2022，58（9）：9-18.
GAO G S.Survey on attention mechanismsin deep learning recommendation models[J].Computer Engineering and Applications，2022，58（9）：9-18.
[29] ZHAO H，ZHANG Y，LIU S，et al.PSANet：point-wise spatial attention network for scene parsing[C]//Proceedings of the 15th European Conference on Computer Vision，2018：267-283.
[30] FU J，LIU J，TIAN H，et al.Dual attention network for scene segmentation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：3146-3154.
[31] NIU R，SUN X，TIAN Y，et al.Hybrid multiple attention network for semantic segmentation in aerial images[J].IEEE Transactions on Geoscience and Remote Sensing，2021，60：1-18.
[32] HOU Q，ZHOU D，FENG J.Coordinate attention for efficient mobile network design[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：13713-13722.
[33] YADAV G，MAHESHWARI S，AGARWAL A.Contrast limited adaptive histogram equalization based enhancement for real time video system[C]//Proceedings of the 2014 International Conference on Advances in Computing，Communications and Informatics，2014：2392-2397.