Rail Surface Defect Method Based on Bimodal-Modal Deep Learning

doi:10.3778/j.issn.1002-8331.2209-0364

Abstract

Abstract: Aiming at the problem that the current rail surface detection system lacks the third dimension key depth information and the detection results are vulnerable to interference with high false alarm rate, the paper designs a rail surface defect detection system and method based on dual mode structured light sensor. By constructing a multi-mode depth learning detection network for rail surface defects, the defects in the bimodal rail images can be detected. The depth network proposed fuses multi-scale features of bimodal images respectively, and conducts multi-scale rail head surface defect detection. Experimental results show that this method can significantly reduce false positives while maintaining a high detection rate. Compared with the common depth learning detection model in the current defect detection, the mean average accuracy（mAP） has been greatly improved, and its performance is better than the previous detection algorithm, which has a good application prospect in the rail head surface scratch detection task.

Key words: defect detection, multi-modal, deep learning, rail head surface

摘要： 针对目前钢轨顶面擦伤检测系统缺少第三维关键深度信息，检测结果易受干扰误报率高的问题，提出了一种基于双模态结构光传感器的钢轨表面缺陷检测方法。通过构建轨道表面缺陷的多模态深度学习检测网络，可以检测双模态钢轨图像中的擦伤缺陷。提出的深度网络分别融合了双模态图像的多尺度特征，并进行多尺度钢轨顶面擦伤检测。实验结果表明，该方法在显著降低检测误报的同时能够保持较高的检出率。与当前缺陷检测中常见的深度学习检测模型对比，平均精度均值（mAP）有大幅提升，性能优于以往的检测算法，在钢轨顶面擦伤检测任务中的应用前景良好。

关键词: 词缺陷检测, 多模态, 深度学习, 钢轨表面

ZHAO Hongwei, ZHENG Jiajun, ZHAO Xinxin, WANG Shengchun, LI Yidong. Rail Surface Defect Method Based on Bimodal-Modal Deep Learning[J]. Computer Engineering and Applications, 2023, 59(7): 285-293.

赵宏伟, 郑嘉俊, 赵鑫欣, 王胜春, 李浥东. 基于双模态深度学习的钢轨表面缺陷检测方法[J]. 计算机工程与应用, 2023, 59(7): 285-293.

References

[1] 王江平，吕金洪，李士刚.用于轨道的无损检测调节装置：CN212579858U[P].2021.
WANG J P，LU J H，LI S G.Nondestructive testing regulation device for track：CN212579858U[P].2021.
[2] 严俨，刘怡，朱俊臻，等.基于电涡流脉冲热成像偏度特征的钢轨自然裂纹量化评估[J].机械工程学报，2021，57（18）：75-85.
YAN Y，LIU Y，ZHU J Z，et al.Quantification of rolling contact fatigue cracks in rails based on eddy current pulsed thermography with skewness features[J].Journal of Mechanical Engineering，2021，57（18）：75-85.
[3] 马骞，陈智发，云维锐.基于相控阵超声技术的重载铁路钢轨焊缝检测方法[J].铁道建筑，2022（2）：47-52.
MA Q，CHEN Z F，YUN W R.Detection method of heavy haul railway rail welding seam based on phased array ultrasonic technology[J].Railway Engineering，2022（2）：47-52.
[4] 陈伟.高速铁路无砟轨道主要病害分析与无损检测[J].设备管理与维修，2020（6）：91-92.
CHEN W.Main disease analysis and nondestructive testing of ballastless track in high-speed railway[J].Plant Maintenance Engineering，2020（6）：91-92.
[5] 吴福培，魏亚辉，李庆华，等.高速轨道超声成像伤损检测及其参数学习方法[J].计算机集成制造系统，2021，27（3）：347-356
WU F P，WEI Y H，LI Q H，et al.Damage detection and parameter learning method of high-speed track ultrasonic imaging[J].Computer Integrated Manufacturing Systems，2021，27（3）：347-356.
[6] 胡文博，邱实，许馨月，等.基于深度学习的钢轨伤损超声检测与分类[J].铁道学报，2021，43（4）：108-116.
HU W B，QIU S，XU X Y，et al.Ultrasonic detection and classification of rail damage based on deep learning[J].Journal of China Railway Society，2021，43（4）：108-116.
[7] 李忠海，白秋阳，王富明，等.基于语义分割的钢轨表面缺陷实时检测系统[J].计算机工程与应用，2021，57（12）：248-256.
LI Z H，BAI Q Y，WANG F M，et al.Real-time detection system of rail surface defects based on semantic segmentation[J].Computer Engineering and Applications，2021，57（12）：248-256.
[8] 罗晖，徐广隆.基于图像增强与深度学习的钢轨表面缺陷检测[J].铁道科学与工程学报，2021，18（3）：623-629.
LUO H，XU G L.Rail surface defect detection based on image enhancement and deep learning[J].Journal of Railway Science and Engineering，2021，18（3）：623-629.
[9] 王超，杨家旺，何建波，等.智能化铁轨探伤科技创新性研究[J].科学技术创新，2021（5）：118-119.
WANG C，YANG J W，HE J B，et al.Innovative research on intelligent rail flaw detection technology[J].Science and Technology Innovation，2021（5）：118-119.
[10] 张唯，李熙，姚德臣，等.基于计算机视觉的轨道列车维护检测装置：CN112129777A[P].2020.
ZHANG W，LI X，YAO D C，et al.Railway train maintenance and detection device based on computer vision：CN112129777A[P].2020.
[11] 贾强，韩峰.基于改进二维卷积滤波技术的轨道伤损检测方法[J].城市轨道交通研究，2022，25（4）：181-185.
JIA Q，Han F.Track damage detection method based on improved two-dimensional convolution filtering technology[J].Urban Mass Transit，2022，25（4）：181-185.
[12] HUBER-M?RK R，N?LLE M，OBERHAUSER A，et al.Statistical rail surface classification based on 2D and 21/2D image analysis[C]//Proceedings of International Conference on Advanced Concepts for Intelligent Vision Systems，2010：50-61.
[13] LI Q，REN S.A real-time visual inspection system for discrete surface defects of rail heads[J].IEEE Transactions on Instrumentation and Measurement，2012，61（8）：2189-2199.
[14] LI Q，REN S.A visual detection system for rail surface defects[J].IEEE Transactions on Systems Man and Cybernetics Part C（Applications and Reviews），2012，42（6）：1531-1542.
[15] YU H，LI Q，TAN Y，et al.A coarseto-fine model for rail surface defect detection[J].IEEE Transactions on Instrumentation and Measurement，2019，68（3）：656-666.
[16] GAN J，LI Q，WANG J，et al.A hierarchical extractor-based visual rail surface inspection system[J].IEEE Sensors Journal，2017，17（23）：7935-7944.
[17] FAGHIH-ROOHI S，SIAMAK H，NUNEZ A，et al.Deep convolutional neural networks for detection of rail surface defects[C]//Proceedings of International Joint Conference on Neural Networks（IJCNN 2016），2016：2584-2589.
[18] SHANG L，YANG Q，WANG J，et al.Detection of rail surface defects based on CNN image recognition and classification[C]//Proceedings of 2018 20th International Conference on Advanced Communication Technology（ICACT），2018：45-51.
[19] HE Y，SONG K，MENG Q，et al.An end-to-end steel surface defect detection approach via fusing multiple hierarchical features[J].IEEE Transactions on Instrumentation and Measurement，2020，69（4）：1493-1504.
[20] 徐科，杨朝霖，周鹏，等.基于激光线光源的钢轨表面缺陷三维检测方法[J].机械工程学报，2010，46（8）：1-5.
XU K，YANG Z L，ZHOU P，et al.3D detection technique of surface defects for steel rails based on linear lasers[J].Journal of Mechanical Engineering，2010，46（8）：1-5.
[21] LI P，WANG P J，CHEN P，et al.Rail corrugation detection based on 3D structured light and wavelet analysis[J].Railway Standard Design，2018，62（8）：33-38.
[22] 周娟，李勇平，黄跃峰.基于强度图和深度图的多模态人脸识别[J].计算机工程与应用，2012，48（25）：5-8.
ZHOU J，LI Y P，HUANG Y F.Multi-modal face recognition based on intensity image and depth image[J].Computer Engineering and Applications，2012，48（25）：5-8.
[23] 宋冠军，张树东，卫飞高.音视频双模态情感识别融合框架研究[J].计算机工程与应用，2020，56（6）：140-146.
SONG G J，ZHANG S D，WEI F G.Research on audio-visual dual-modal emotion recognition fusion frame-work[J].Computer Engineering and Applications，2020，56（6）：140-146.
[24] 潘梦竹，李千目，邱天.深度多模态表示学习的研究综述[J].计算机工程与应用，2023，59（2）：48-64.
PAN M Z，LI Q M，QIU T.Survey of research on deep multimodal representation learning[J].Computer Engineering and Applications，2023，59（2）：48-64.
[25] DAI J，QI H，XIONG Y，et al.Deformable convolutional networks[C]//Proceedings of 2017 IEEE International Conference on Computer Vision（ICCV），2017：764-773.
[26] WANG J，XU C，YANG Z，et al.Deformable convolutional networks for efficient mixed-type wafer defect pattern recognition[J].IEEE Transactions on Semiconductor Manufacturing，2020，33（4）：587-596.
[27] YANAN S，HUI Z，LI L，et al.Rail surface defect detection method based on yolov3 deep learning networks[C]//Proceedings of 2018 Chinese Automation Congress（CAC），2019：1563-1568.
[28] BOCHKOVSKIY A，WANG C Y，LIAO H Y M.YOLOv4：optimal speed and accuracy of object detection[J].arXiv：2004.10934，2020.