基于多记忆增强模块及图像轮廓重建的工业表面异常检测

doi:10.3778/j.issn.1002-8331.2406-0293

摘要/Abstract

摘要： 基于重建的工业图像异常检测通常假设模型能很好重建正常区域，而不能很好重建异常区域。但由于深度神经网络存在过度泛化问题，使得异常区域也能被较好重建，导致异常区域漏检。为解决上述问题，提出一种基于多记忆增强模块及图像轮廓重建的工业表面异常检测网络（industrial surface anomaly detection based on reconstruction with multiple memory enhancement modules and image edge，MMAERec）。具体来说，在带有跳跃连接的U-Net类型去噪自编码器上引入多记忆增强模块和图像轮廓提取模块。多记忆增强模块得到的记忆特征有利于很好地重建正常区域；而提取到的图像轮廓特征则有利于很好重建图像轮廓。将这两种不同特征的融合经注意力机制处理并用于重建能很好地提高重建图像质量。所提方法可以强制网络学习正常的低频和高频信息，防止模型直接复制异常区域，有效缓解过度泛化问题。在MVTec AD和BTAD两个工业数据集上的实验结果也展现了所提方法良好的检测和定位性能。

关键词: 重建网络, 多记忆增强模块, 注意力机制, 图像轮廓

Abstract: Reconstruction-based anomaly detection in industrial images usually assumes that the model can reconstruct the normal region well, but not the abnormal region well. However, due to the over-generalization problem of deep neural networks, abnormal regions can also be reconstructed well, resulting in the leakage of abnormal regions. In order to solve the above problems, this paper proposes an industrial surface anomaly detection model based on reconstruction with multiple memory enhancement modules and image edge (MMAERec). Specifically, multiple memory enhancement modules and image edge extraction modules are introduced on a UNet-type denoising self-encoder with skip connections. The memory features obtained from the multi-memory enhancement module facilitate a good reconstruction of the normal region, while the extracted image edge features facilitate a good reconstruction of the image contour. The fusion of these two different features processed by the attention mechanism and used for reconstruction can improve the quality of the reconstructed image very well. The proposed method can force the network to learn normal low-frequency and high-frequency information, preventing the model from directly replicating the abnormal regions, and effectively alleviating the overgeneralization problem. Experimental results on two industrial datasets, MVTec AD and BTAD, also demonstrate the good detection and localization performance of the proposed method.

Key words: reconstruction network, multiple memory enhancement modules, attentional mechanism, image edge

杨曜, 许湘云, 张琳娜, 陈建强, 岑翼刚, 黄彦森. 基于多记忆增强模块及图像轮廓重建的工业表面异常检测[J]. 计算机工程与应用, 2025, 61(20): 248-259.

YANG Yao, XU Xiangyun, ZHANG Linna, CHEN Jianqiang, CEN Yigang, HUANG Yansen. Industrial Surface Anomaly Detection Based on Reconstruction with Multiple Memory Enhancement Modules and Image Edge[J]. Computer Engineering and Applications, 2025, 61(20): 248-259.

参考文献

[1] ZHANG F H, KAN S C, ZHANG D M, et al. A graph model-based multiscale feature fitting method for unsupervised anomaly detection[J]. Pattern Recognition, 2023, 138: 109373.
[2] ZHANG L Y, KAN S C, CEN Y G, et al. A normalizing flow-based bidirectional mapping residual network for unsupervised defect detection[J]. Computers, Materials & Continua, 2024, 78(2): 1631-1648.
[3] 张兰尧, 陈晓玲, 张达敏, 等. ValidFlow: 基于标准化流的无监督图像缺陷检测[J].数据采集与处理, 2023, 38(6): 1445-1457.
ZHANG L, CHEN X, ZHANG D, et al. ValidFlow: unsupervised image defect detection based on normalizing flows[J].Journal of Data Acquisition and Processing, 2023, 38(6): 1445-1457.
[4] DEFARD T, SETKOV A, LOESCH A, et al. PaDiM: a patch distribution modeling framework for anomaly detection and localization[C]//Proceedings of the International Conference on Pattern Recognition. Cham: Springer International Publi-shing, 2021: 475-489.
[5] ZHOU Y, XU X, SONG J, et al. MSFlow: multiscale flow-based framework for unsupervised anomaly detection[J]. IEEE Transactions on Neural Networks and Learning Systems, 2025, 36(2): 2437-2450.
[6] COHEN N, HOSHEN Y. Sub-image anomaly detection with deep pyramid correspondences[J]. arXiv:2005.02357, 2020.
[7] WANG G D, HAN S M, DING E R, et al. Student-teacher feature pyramid matching for anomaly detection[J]. arXiv: 2103.04257, 2021.
[8] YANG J, SHI Y, QI Z Q. DFR: deep feature reconstruction for unsupervised anomaly segmentation[J]. arXiv:2012.07122, 2020.
[9] GONG D, LIU L Q, LE V, et al. Memorizing normality to det-ect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 1705-1714.
[10] YAMADA S, KAMIYA S, HOTTA K. Reconstructed student-teacher and discriminative networks for anomaly dete-ction[C]//Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway: IEEE, 2022: 2725-2732.
[11] DENG J, DONG W, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2009: 248-255.
[12] LIU T, LI B, ZHAO Z, et al. Reconstruction from edge image combined with color and gradient difference for industrial surface anomaly detection[J]. arXiv:2210.14485, 2022.
[13] BERGMANN P, FAUSER M, SATTLEGGER D, et al. MVTec AD: a comprehensive real-world dataset for unsupervised anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 9584-9592.
[14] MISHRA P, VERK R, FORNASIER D, et al. VT-ADL: a vision transformer network for image anomaly detection and localization[C]//Proceedings of the IEEE 30th International Symposium on Industrial Electronics. Piscataway: IEEE, 2021: 1-6.
[15] RUFF L, VANDERMEULEN R, GOERNITZ N, et al. Deep one-class classification[C]//Proceedings of the International Conference on Machine Learning, 2018: 4393-4402.
[16] BERGMAN L, HOSHEN Y. Classification-based anomaly detection for general data[J]. arXiv:2005.02359, 2020.
[17] RIPPEL O, MERTENS P, MERHOF D. Modeling the distribution of normal data in pre-trained deep features for anomaly detection[C]//Proceedings of the 25th International Conference on Pattern Recognition. Piscataway: IEEE, 2021: 6726-6733.
[18] ZAVRTANIK V, KRISTAN M, SKO?AJ D. Reconstruction by inpainting for visual anomaly detection[J]. Pattern Recognition, 2021, 112: 107706.
[19] BERGMANN P, L?WE S, FAUSER M, et al. Improving unsupervised defect segmentation by applying structural similarity to auto-encoders[J]. arXiv:1807.02011, 2018.
[20] COLLIN A S, DE VLEESCHOUWER C. Improved anomaly detection by training an autoencoder with skip connections on images corrupted with Stain-shaped noise[C]//Proceedings of the 25th International Conference on Pattern Recognition. Piscataway: IEEE, 2021: 7915-7922.
[21] VENKATARAMANAN S, PENG K C, SINGH R V, et al. Attention guided anomaly localization in images[C]//Proceedings of the European Conference on Computer Vision.Cham: Springer International Publishing, 2020: 485-503.
[22] LIANG Y, ZHANG J, ZHAO S, et al. Omni-frequency channel-selection representations for unsupervised anomaly detection[J]. IEEE Trans Image Process, 2023, 32: 4327-4340.
[23] SCHLEGL T, SEEB?CK P, WALDSTEIN S M, et al. F-AnoGAN: fast unsupervised anomaly detection with generative adversarial networks[J].Medical Image Analysis, 2019, 54: 30-44.
[24] MEI S, YANG H, YIN Z P. An unsupervised-learning-based approach for automated defect inspection on textured surfaces[J]. IEEE Transactions on Instrumentation and Measurement, 2018, 67(6): 1266-1277.
[25] KANG G Q, GAO S B, YU L, et al. Deep architecture for high-speed railway insulator surface defect detection: deno-ising autoencoder with multitask learning[J]. IEEE Transa-ctions on Instrumentation and Measurement, 2019, 68(8): 2679-2690.
[26] GOLAN I, EL-YANIV R. Deep anomaly detection using geometric transformations[J]. arXiv:1805.10917, 2018.
[27] YE F, HUANG C Q, CAO J K, et al. Attribute restoration framework for anomaly detection[J]. IEEE Transactions on Multimedia, 2022, 24: 116-127.
[28] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Trans Image Process, 2004, 13(4): 600-612.
[29] CWOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision. New York: ACM, 2018: 3-19.
[30] BERGMANN P, FAUSER M, SATTLEGGER D, et al. Uninformed students: student-teacher anomaly detection with discriminative latent embeddings[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE, 2020: 4182-4191.
[31] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141.
[32] FUKUI H, HIRAKAWA T, YAMASHITA T, et al. Attention branch network: learning of attention mechanism for visual explanation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE, 2019: 10697-10706.
[33] HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE, 2021: 13708-13717.
[34] SHI Y, YANG J, QI Z. Unsupervised anomaly segmentation via deep feature reconstruction[J]. Neurocomputing, 2021, 424: 9-22.
[35] SCHLüTER H M, TAN J, HOU B . Self-supervised out-of-distribution detection and localization with natural synthetic anomalies (NSA)[J]. arXiv:2109.15222, 2021.
[36] YI J , YOON S. Patch SVDD: patch-level SVDD for anomaly detection and segmentation[C]//Proceedings of the Asian Conference on Computer Vision, 2020: 375-390.
[37] PIRNAY J, CHAI K. Inpainting transformer for anomaly detection[C]//Proceedings of the International Conference on Image Analysis and Processing. Cham: Springer International Publishing, 2022: 394-406.
[38] HOU J L, ZHANG Y Y, ZHONG Q Y, et al. Divide-and-assemble: learning block-wise memory for unsupervised anomaly detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 8771-8780.
[39] ROTH K, PEMULA L, ZEPEDA J, et al. Towards total recall in industrial anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE, 2022: 14298-14308.