融合扩散模型的生成式零样本钢表面缺陷检测

doi:10.3778/j.issn.1002-8331.2407-0368

摘要/Abstract

摘要： 针对生成式零样本目标检测模型难以应对复杂场景下的钢材表面缺陷检测，存在语义混淆和鲁棒性低的问题，提出一种融合扩散模型的生成式零样本钢材表面缺陷检测模型。设计多模态缺陷特征对齐模块，通过监督对比学习、缺陷特征对齐和语义一致性重建，使生成器生成的缺陷特征与原始语义信息充分对齐，提高生成模型的鲁棒性；引入缺陷特征去噪扩散模块，通过逐步添加、去除噪声来生成多样化的特征表征，并筛选出具有代表性的生成缺陷特征。将得到的生成缺陷特征用于更新缺陷检测网络的分类器，实现零样本钢材表面缺陷检测。通过在NEU和GC10两个钢材表面缺陷数据集上的实验结果显示，零样本检测设置下，检测精度相较于基线模型分别提升11.5和17.4个百分点；广义零样本检测设置下，调和平均值分别提升3.0和9.8个百分点，有效提升了模型在复杂场景下的钢材表面缺陷检测能力；可视化结果表明，模型能够生成分离特征明显的未见缺陷特征，缓解了语义混淆问题；此外，与目前先进的零样本目标检测模型相比，该模型在钢材表面缺陷检测中表现出了更高的准确率和鲁棒性。

关键词: 缺陷检测, 零样本学习, 生成式模型, 语义对齐, 扩散模型

Abstract: To address the challenges of semantic confusion and low robustness in generative zero-shot target detection models for complex scenes in steel surface defect detection, a generative zero-shot steel surface defect detection model incorporating a diffusion model is proposed. A multi-modal defect feature alignment module is designed to fully align the defect features generated by the generator with the original semantic information through supervised contrastive learning, defect feature alignment, and semantic consistency reconstruction, thereby improving the robustness of the generative model. A defect feature denoising diffusion module is introduced to generate diverse feature representations by gradually adding and removing noise, and representative generated defect features are selected. The obtained generated defect features are used to update the classifier of the defect detection network, achieving zero-shot steel surface defect detection. Experiments conducted on the NEU and GC10 steel surface defect datasets show that under the zero-shot detection setting, the detection accuracy is improved by 11.5 and 17.4 percentage points, respectively, compared to baseline models. Under the generalized zero-shot detection setting, the harmonic mean is improved by 3.0 and 9.8 percentage points, respectively, effectively enhancing the capability of the model in detecting steel surface defects in complex scenes. Visualization results indicate that the model can generate unseen defect features with distinct characteristics, alleviating semantic confusion. Additionally, compared with current advanced zero-shot detection models, this model exhibits higher accuracy and robustness in steel surface defect detection.

Key words: defect detection, zero-shot learning, generative model, semantic alignment, diffusion model

季瑞瑞, 杨思凡, 华羽垚, 耿屹, 白晨羲. 融合扩散模型的生成式零样本钢表面缺陷检测[J]. 计算机工程与应用, 2025, 61(12): 333-343.

JI Ruirui, YANG Sifan, HUA Yuyao, GENG Yi, BAI Chenxi. Generative Zero-Shot Steel Surface Defect Detection Integrating Diffusion Model[J]. Computer Engineering and Applications, 2025, 61(12): 333-343.

参考文献

[1] ABDO A M A N, CZúNI L. Zero-shot learning and classification of steel surface defects[C]//Proceedings of the Fourteenth International Conference on Machine Vision, 2022: 45.
[2] HE Z Q, LIU Q F. Deep regression neural network for industrial surface defect detection[J]. IEEE Access, 2020, 8: 35583-35591.
[3] GUO Y B, FAN Y M, XIANG Z Y, et al. Zero-sample surface defect detection and classification based on semantic feedback neural network[J]. arXiv:2106.07959, 2021.
[4] HABIB M T, SHUVO S B, UDDIN M S, et al. Automated textile defect classification by Bayesian classifier based on statistical features[C]//Proceedings of the 2016 International Workshop on Computational Intelligence. Piscataway: IEEE, 2016: 101-105.
[5] YU J B, LU X L. Wafer map defect detection and recognition using joint local and nonlocal linear discriminant analysis[J]. IEEE Transactions on Semiconductor Manufacturing, 2016, 29(1): 33-43.
[6] GROCHOWALSKI J M, CHADY T. Rapid identification of material defects based on pulsed multifrequency eddy current testing and the k-nearest neighbor method[J]. Materials, 2023, 16(20): 6650.
[7] SOUKUP D, HUBER-M?RK R. Convolutional neural networks for steel surface defect detection from photometric stereo images[C]//Proceedings of the International Symposium on Visual Computing. Cham: Springer International Publishing, 2014: 668-677.
[8] WEI R B, SONG Y H, ZHANG Y L. Enhanced faster region convolutional neural networks for steel surface defect detection[J]. ISIJ International, 2020, 60(3): 539-545.
[9] WANG L, LIU X B, MA J T, et al. Real-time steel surface defect detection with improved multi-scale YOLO-v5[J]. Processes, 2023, 11(5): 1357.
[10] LAMPERT C H, NICKISCH H, HARMELING S. Learning to detect unseen object classes by between-class attribute transfer[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2009: 951-958.
[11] LI Z, GAO L, GAO Y, et al. Zero-shot surface defect recognition with class knowledge graph[J]. Advanced Engineering Informatics, 2022, 54: 101813.
[12] FENG L J, ZHAO C H. Fault description based attribute transfer for zero-sample industrial fault diagnosis[J]. IEEE Transactions on Industrial Informatics, 2021, 17(3): 1852-1862.
[13] GAO J Y, ZHANG T Z, XU C S. I know the relationships: zero-shot action recognition via two-stream graph convolutional networks and knowledge graphs[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019: 8303-8311.
[14] RAHMAN S, KHAN S, BARNES N. Improved visual-semantic alignment for zero-shot object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 11932-11939.
[15] CHEN J L, PAN T Y, ZHOU Z T, et al. An adversarial learning framework for zero-shot fault recognition of mechanical systems[C]//Proceedings of the 2019 IEEE 17th International Conference on Industrial Informatics. Piscataway: IEEE, 2019: 1275-1278.
[16] LV H X, CHEN J L, PAN T Y, et al. Hybrid attribute conditional adversarial denoising autoencoder for zero-shot classification of mechanical intelligent fault diagnosis[J]. Applied Soft Computing, 2020, 95: 106577.
[17] HAYAT N, HAYAT M, RAHMAN S, et al. Synthesizing the unseen for zero-shot object detection[C]//Proceedings of the Asian Conference on Computer Vision. Cham: Springer, 2021: 155-170.
[18] SARMA S, KUMAR S, SUR A. Resolving semantic confusions for improved zero-shot detection[J]. arXiv:2212.06097, 2022.
[19] HUANG P L, HAN J W, CHENG D, et al. Robust region feature synthesizer for zero-shot object detection[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 7612-7621.
[20] 赵楚, 段先华, 苏俊楷. 改进Faster RCNN的瓷砖表面瑕疵检测研究[J]. 计算机工程与应用, 2023, 59(14): 201-208.
ZHAO C, DUAN X H, SU J K. Research on ceramic tile surface defect detection by improved Faster RCNN[J]. Computer Engineering and Applications, 2023, 59(14): 201-208.
[21] ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein generative adversarial networks[C]//Proceedings of the International Conference on Machine Learning, 2017: 214-223.
[22] HE X J, LUO Z Q, LI Q Y, et al. DG-GAN: a high quality defect image generation method for defect detection[J]. Sensors, 2023, 23(13): 5922.
[23] JAIN S, SETH G, PARUTHI A, et al. Synthetic data augmentation for surface defect detection and classification using deep learning[J]. Journal of Intelligent Manufacturing, 2022, 33(4): 1007-1020.
[24] KHOSLA P, TETERWAK P, WANG C, et al. Supervised contrastive learning[J]. arXiv:2004.11362, 2020.
[25] LIU R, GE Y X, CHOI C L, et al. DivCo: diverse conditional image synthesis via contrastive generative adversarial network[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 16372-16381.
[26] SCHROFF F, KALENICHENKO D, PHILBIN J. FaceNet: a unified embedding for face recognition and clustering[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 815-823.
[27] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 2242-2251.
[28] ZHOU P F, MIN W Q, ZHANG Y, et al. SeeDS: semantic separable diffusion synthesizer for zero-shot food detection[J]. arXiv:2310.04689, 2023.
[29] 周成龙, 陈玉明, 朱益冬. 粒K均值聚类算法[J]. 计算机工程与应用, 2023, 59(13): 317-324.
ZHOU C L, CHEN Y M, ZHU Y D. Granular K-means clustering algorithm[J]. Computer Engineering and Applications, 2023, 59(13): 317-324.
[30] SONG K C, YAN Y H. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects[J]. Applied Surface Science, 2013, 285: 858-864.
[31] 苏佳, 贾泽, 秦一畅, 等. 面向工业表面缺陷检测的改进YOLOv8算法[J]. 计算机工程与应用, 2024, 60(14): 187-196.
SU J, JIA Z, QIN Y C, et al. Improved YOLOv8 algorithm for industrial surface defect detection[J]. Computer Engineering and Applications, 2024, 60(14): 187-196.
[32] HAN Z Y, FU Z Y, CHEN S, et al. Contrastive embedding for generalized zero-shot learning[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 2371-2381.
[33] DEMIREL B, CINBIS R G, IKIZLER-CINBIS N. Zero-shot object detection by hybrid region embedding[J]. arXiv:1805. 06157, 2018 .
[34] ZHENG Y, HUANG R R, HAN C Q, et al. Background learnable cascade for zero-shot object detection[C]//Proceedings of the Asian Conference on Computer Vision. Cham: Springer, 2021: 107-123.
[35] RAHMAN S, KHAN S, BARNES N. Polarity loss for zero-shot object detection[J]. arXiv:1811.08982, 2018.