Review of Acoustic Signal-Based Industrial Equipment Fault Diagnosis

doi:10.3778/j.issn.1002-8331.2208-0079

Abstract

Abstract: In order to guarantee the safety and stability of the industrial production process, it is of great significance and value to adopt reasonable fault diagnosis. Thus, fault diagnosis of industrial equipment has always been a hotspot in the field of industrial control. Firstly, this paper discusses the significance of fault diagnosis, and points out the feasibility and advantages of fault diagnosis based on acoustic signal. Then, according to whether the deep learning is involved, acoustic signal-based fault diagnosis approaches are segmented into traditional-based and deep learning-based categories. Then, it combs the essential ideas and flow of two categories respectively, expounds and summarizes the principle, advantages, limitations, main methods and diagnostic results. Finally, the paper points out the research difficulties, hotspots and the future development direction in the area of industrial equipment fault diagnosis.

Key words: acoustic signal, fault diagnosis, industrial equipment, machine learning, deep learning

摘要： 为了保证工业生产过程的安全稳定运行，采取合理的故障诊断具有十分重要的意义和价值。因此，工业设备故障诊断一直是工业领域的研究热点。阐述了故障诊断的意义，并指出基于声信号进行故障诊断的可行性和优势。根据有无深度学习的参与，将基于声信号的故障诊断方法分为基于传统和基于深度学习两种类型；分别梳理了两类故障诊断方法的流程与思路，阐述并归纳了两类方法中典型算法的原理、优点、局限性、主要方法及诊断效果。最后，指出了当前工业设备故障诊断领域的研究难点、热点以及未来发展方向。

关键词: 声信号, 故障诊断, 工业设备, 机器学习, 深度学习

ZHOU Yurong, ZHANG Qiaoling, YU Guangzeng, XU Weiqiang. Review of Acoustic Signal-Based Industrial Equipment Fault Diagnosis[J]. Computer Engineering and Applications, 2023, 59(7): 51-63.

周玉蓉, 张巧灵, 于广增, 徐伟强. 基于声信号的工业设备故障诊断研究综述[J]. 计算机工程与应用, 2023, 59(7): 51-63.

References

[1] 柴天佑.工业人工智能发展方向[J].自动化学报，2020，46（10）：2005-2012.
CHAI T Y.Development directions of industrial artificial intelligence[J].Acta Automatica Sinica，2020，46（10）：2005-2012.
[2] HOU Z，CHI R，GAO H.An overview of dynamic-linearization-based data-driven control and applications[J].IEEE Transactions on Industrial Electronics，2017，64（5）：4076-4090.
[3] CHOI S，HAQUE M S，TAREK M T B，et al.Fault diagnosis techniques for permanent magnet AC machine and drives—a review of current state of the art[J].IEEE Transactions on Transportation Electrification，2018，4（2），444-463.
[4] ZHOU D，ZHAO Y，WANG Z，et al.Review on diagnosis techniques for intermittent faults in dynamic systems[J].IEEE Transactions on Industrial Electronics，2019，64（3）：2337-2347.
[5] 文成林，吕菲亚.基于深度学习的故障诊断方法综述[J].电子与信息学报，2020，42（1）：234-248.
WEN C L，LYU Y F.Review on deep learning based fault diagnosi[J].Journal of Electronics & Information Technology，2020，42（1）：234-248.
[6] DELVECCHIO S，BONFIGLIO P，POMPOLI F.Vibro-acoustic condition monitoring of internal combustion engines：a critical review of existing techniques[J].Mechanical Systems and Signal Processing，2017，99（15）：661-683.
[7] MOUZAKITIS A.Classification of fault diagnosis methods for control systems[J].Measurement & Control，2013，46（10）：303-308.
[8] GAO Z，CECATI C，DING S X.A survey of fault diagnosis and fault-tolerant techniques—part I：fault diagnosis with model-based andsignal-based approaches[J].IEEE Transactions on Industrial Electronics，2015，62（6）：3757-3767.
[9] SUN T D，YU G，GAO M，et al.Fault diagnosis methods based on machine learning and its applications for wind turbines：a review[J].IEEE Access，2021，9：147481-147511.
[10] ZHANG S H，ZHOU J H，WANG E H，et al.State of the art on vibration signal processing towards data-driven gear fault diagnosis[J].IET Collaborative Intelligent Manufacturing，2022，4（4）：249-266.
[11] WANG B，DONG M，WU Z Y，et al.Automatic fault diagnosis of infrared insulator images based on image instance segmentation and temperature analysis[J].IEEE Transactions on Instrumentation and Measurement，2020，69（8）：5345-5355.
[12] LIU T，LI G，GAO Y.Fault diagnosis method of substation equipment based on you only look once algorithm and infrared imaging[J].Energy Reports，2022，8：171-180.
[13] 李伟，李硕.理解数字声音——基于一般音频/环境声的计算机听觉综述[J].复旦学报（自然科学版），2019，58（3）：269-313.
LI W，LI S.Understanding digital audio—a review of general audio/ambient sound based computer audition[J].Journal of Fudan University（Natural Science），2019，58（3）：269-313.
[14] 曹昳劼.基于压力、振动、声音信号的压气机喘振故障诊断和监测[D].上海：上海交通大学，2010.
CAO Y J.Diagnosisand monitoring of the compressor surge based on the pressure，vibration and sound signals[D].Shanghai：Shanghai Jiaotong University，2010.
[15] SAUFI S R，AHMAD Z A B，LEONG M S，et al.Low-speed bearing fault diagnosis based on ArSSAE model using acoustic emission and vibration signals[J].IEEE Access，2019，7：46885-46897.
[16] SHARMA V，PAREY A.Case study on the effectiveness of gear fault diagnosis technique for gear tooth defects under fluctuating speed[J].IET Renewable Power Generation，2017，11（14）：1841-1849.
[17] RAGHAV M S，SHARMA R B.A Review on fault diagnosis and condition monitoring of gearboxes by using AE technique[J].Archives of Computational Methods in Engineering，2020，28（4）：2845-2859.
[18] SHEVCHIK S A，ZANOLI S，SAEIDI F，et al.Monitoring of friction-related failures using diffusion maps of acoustic time series[J].Mechanical Systems and Signal Processing，2021，148：107172.
[19] LEVIKARI S，K?RKK?INEN T J，ANDERSSON C，et al.Acoustic phenomena in damaged ceramic capacitors[J].IEEE Transactions on Industrial Electronics，2017，65（1）：570-577.
[20] ZHANG J，HU X，ZHONG X，et al.Fault diagnosis of axle box bearing with acoustic signal based on chirplet transform and support vector machine[J].Shock and Vibration，2022：9868999.
[21] CHOE C Y，CHEN C T，NAGAO S，et al.Real-time acoustic emission monitoring of wear-out failure in SiC power electronic devices during power cycling tests[J].IEEE Transactions on Power Electronics，2020，36（4）：4420-4428.
[22] MARAABA L S，MEMON A M，ABIDO M A，et al.An efficient acoustic-based diagnosis of inter-turn fault in interior mount LSPMSM[J].Applied Acoustics，2021，173：107661.
[23] ALBARBAR A，GU F，BALL A D，et al.Acoustic monitoring of engine fuel injection based on adaptive filtering techniques[J].Applied Acoustics，2010，71（12）：1132-1141.
[24] ALBARBAR A，GU F，BALL A D.Diesel engine fuel injection monitoring using acoustic measurements and independent component analysis[J].Measurement，2010，43（10）：1376-1386.
[25] HUANG R R，YAN L，LIU J.Laser GYRO signal filtering by combining CEEMDAN and principal component analysis[J].Journal of Vibro Engineering，2021，23（8）：1820-1832.
[26] 谯自健，束学道.非对称势诱导随机共振增强机械重复瞬态提取[J].机械工程学报，2021，57（23）：160-168.
QIAO Z J，SHU X D.Stochastic resonance induced by asymmetric potentials enhanced mechanical repetitive transient extraction[J].Journal of Mechanical Engineering，2021，57（23）：160-168.
[27] YAO J C，LIU C，SONG K Y，et al.Fault diagnosis of planetary gearbox based on acoustic signals[J].Applied Acoustics，2021，181：108151.
[28] 张进，吴健，欧习洋，等.基于特征降维和神经网络的电能表内异物声音自动识别[J].机械设计与制造，2021（3）：234-237.
ZHANG J，WU J，OU X Y，et al.Automatic recognition of foreign object sound in the electricity meters based on feature dimension reduction and neural network[J].Machinery Design ＆ Manufacture，2021（3）：234-237.
[29] YU G Y，YAN G，MA B.Feature enhancement method of rolling bearing acoustic signal based on RLS-RSSD[J].Measurement，2022，192：110883.
[30] 杨元威，关永刚，陈士刚，等.基于声音信号的高压断路器机械故障诊断方法[J].中国电机工程学报，2018，38（22）：6730-6737.
YANG Y W，GUAN Y G，CHEN S G，et al.Mechanical fault diagnosis method of high voltage circuit breaker based on sound signal[J].Proceedings of the CSEE，2018，38（22）：6730-6737.
[31] ZHANG L，REN L Q，SHI Y W.A humanoid method for extracting abnormal engine sounds from engine acoustics based on adaptive Volterra filter[J].Journal of Bionic Engineering，2012，9（2）：262-270.
[32] SKOCZYLAS A，STEFANIAK P，ANUFRIIEV S，et al.Belt conveyors rollers diagnostics based on acoustic signal collected using autonomous legged inspection robot[J].Applied Sciences，2021，11（5）：1-13.
[33] LIU X W，PEI D L，LODEWIJKS G，et al.Acoustic signal based fault detection on belt conveyor idlers using machine learning[J].Advanced Powder Technology，2020，31（7）：2689-2698.
[34] 耿琪深，王丰华，金霄.基于Gammatone滤波器倒谱系数与鲸鱼算法优化随机森林的干式变压器机械故障声音诊断[J].电力自动化设备，2020，40（8）：191-199.
GENG Q S，WANG F H，JIN X.Mechanical fault sound diagnosis based on GFCC and random forest optimized by whale algorithm for dry type transformer[J].Electric Power Automation Equipment，2020，40（8）：191-199.
[35] ZHANG Y H，ZHANG K，WANG J，et al.Robust acoustic event recognition using AVMD-PWVD time-frequency image[J].Applied Acoustics，2021，178：107970.
[36] 吕琛，王桂增，邱庆刚.基于声信号小波包分析的故障诊断[J].自动化学报，2004，30（4）：554-559.
LYU C，WANG G Z，QIU Q G.Fault diagnosis based on wavelet packet of sound signal[J].Acta Automatica Sinica，2004，30（4）：554-559.
[37] DELGADO-ARREDONDOP A，MORINIGO-SOTELO D，OSORNIO-RIOS R A，et al.Methodology for fault detection in induction motors via sound and vibration signals[J].Mechanical Systems and Signal Processing，2017，83：568-589
[38] CHEN J Y，FENG Y W，LU C，et al.Fusion fault diagnosis approach to rolling bearing with vibrational and acoustic emission signals[J].CMES-Computer Modeling in Engineering and Sciences，2021，129（2）：1013-1027.
[39] ZHONG K，HAN M，HAN B.Data-driven based fault prognosis for industrial systems：a concise overview[J].IEEE/CAA Journal of Automatica Sinica，2019，7（2）：330-345.
[40] 穆峰.基于声音信号的故障诊断研究及应用[D].济南：山东大学，2016.
MU F.Researchand applications of fault diagnosis based on speech signal[D].Jinan：Shandong University，2016.
[41] YAN M M，WANG X G，WANG B X，et al.Bearing remaining useful life prediction using support vector machine and hybrid degradation tracking model[J].ISA Transactions，2020，98：471-482.
[42] JENA D P，PANIGRAHI S N.Introducing passive acoustic filter in acoustic based condition monitoring：motor bike piston-bore fault identification[J].Mechanical Systems and Signal Processing，2016，70：932-946.
[43] WANG C，LIU C，LIAO M L，et al.An enhanced diagnosis method for weak fault features of bearing acoustic emission signal based on compressed sensing[J].Mathe-matical Biosciences and Engineering，2021，18（2）：1670-1688.
[44] CHEN S，NIU W H，LI B S，et al.Based on EEMD and multiclass relevance vector for high voltage circuit breaker mechanical fault diagnosis[C]//Proceedings of the 3rd International Conference on Energy and Environmental Protection，2014：900-904.
[45] 孙曙光，于晗，杜太行，等.基于多特征融合与改进QPSO-RVM的万能式断路器故障振声诊断方法[J].电工技术学报，2017，32（19）：107-117.
SUN S J，YU H，DU T H，et al.Vibration and acoustic joint fault diagnosis of conventional circuit breaker based on multi-feature fusion and improved QPSO-RVM[J].Transactions of China Electrotechnical Society，2017，32（19）：107-117.
[46] 白翠粉，高文胜，金雷，等.基于3层贝叶斯网络的变压器综合故障诊断[J].高电压技术，2013，39（2）：330-335.
BAI C F，GAO W S，JIN L，et al.Integrated diagnosis of transformer faults based on three-layer Bayesian network[J].High Voltage Engineering，2013，39（2）：330-335.
[47] GUO L，LEI Y G，XING S B，et al.Deep convolutional transfer learning network：a new method for intelligent fault diagnosis of machines with unlabeled data[J].IEEE Transactions on Industrial Electronics，2018，66（9）：7316-7325.
[48] ZHANG G W，JI S S，HAN B K，et al.An intelligent diagnosis method of bearing fault based on acoustic signal processing[C]//Proceedings of the 11th International Conference on Prognostics and System Health Management，2020：137-140.
[49] 陈静.基于声音信号分析的牵引变压器故障诊断方法研究[J].电气应用，2020，39（2）：25-29.
CHEN J.Research on traction transformer fault diagnosis method based on sound signal[J].Electrotechnical Application，2020，39（2）：25-29.
[50] 杨珊.基于异常声音的货运列车滚动轴承故障诊断方法研究[D].长沙：中南大学，2012.
YANG S.Research on fault diagnosis of the freight train rolling bearing based on anomalous sound[D].Changsha：Central South University，2012.
[51] 赵鹏飞.基于音频分析的列车行驶安全检测技术研究[D].天津：天津科技大学，2018.
ZHAO P F.Research on train safety testing technique based on audio analysis[D].Tianjin：Tianjin University of Science & Technology，2018.
[52] MATHEW S K，ZHANG Y.Acoustic-based engine fault diagnosis using WPT，PCA and Bayesian optimization[J].Applied Sciences，2020，10（19）：1-18.
[53] SHANG M S，LUO X，LIU Z G，et al.Randomized latent factor model for high-dimensional and sparse matrices from industrial applications[J].IEEE/CAA Journal of Automatica Sinica，2018，6（1）：131-141.
[54] LEI Y G，JIA F，LIN J，et al.An intelligent fault diagnosis method using unsupervised feature learning towards mechanical big data[J].IEEE Transactions on Industrial Electronics，2016，63（5）：3137-3147.
[55] LEE J，DAVARI H，SINGH J，et al.Industrial artificial intelligence for industry 4.0-based manufacturing systems[J].Manufacturing Letters，2018，18：20-23.
[56] HINTON G E，OSINDERO S，TEH Y W.A fast learning algorithm for deep belief nets[J].Neural Computation，2006，18（7）：1527-1554.
[57] LI L，LI Z N，ZHOU Y，et al.The identification of fatigue crack acoustic emission signal of axle based on depth belief network[C]//Proceedings of the 10th Prognostics and System Health Management Conference，2019.
[58] ZHAO B，WU C J.Sound quality evaluation of electronic expansion valve using Gaussian restricted Boltzmann machines based DBN[J].Applied Acoustics，2020，170：107493.
[59] YUAN N，YANG W，KANG B，et al.RETRACTED：signal fusion-based deep fast random forest method for machine health assessment[J].Journal of Manufacturing Systems，2018，48：1-8.
[60] LI Z W，LIU F，YANG W J，et al.A survey of convolutional neural networks：analysis，applications，and prospects[J].IEEE Transactions on NeuraL Networksand Learning Systems，2021，33（12）：6999-7019.
[61] YAO Y，WANG H L，LI S B，et al.End-to-end convolutional neural network model for gear fault diagnosis based on sound signals[J].Applied Sciences，2018，8（9）.
[62] ZHANG W，LI C H，PENG G L，et al.A deep convolutional neural network with new training methods for bearing fault diagnosis under noisy environment and different working load[J].Mechanical Systems and Signal Processing，2018，100：439-453.
[63] LI X Y，LI J L，HE D，et al.Gear pitting fault diagnosis using raw acoustic emission signal based ondeep learning[J].Eksploatacja i Niezawodnosc，2019，21（3）：403-410.
[64] 姚雪梅.多源数据融合的设备状态监测与智能诊断研究[D].贵阳：贵州大学，2018.
YAO X M.Research on equipment condition monitoring and intelligent diagnosis based on multi-source data fusion[D].Guiyang：Guizhou University，2018.
[65] POLLACK J B.Recursive distributed representations[J].Artificial Intelligence，1990，46（1/2）：77-105.
[66] JIN L，YAN J K，DU X J，et al.RNN for solving time-variant generalized Sylvester equation with applications to robots and acoustic source localization[J].IEEE Transactions on Industrial Informatics，2020，16（10）：6359-6369.
[67] WANG Y，WANG J，SUN J，et al.Investigation on recognition method of acoustic emission signal of the compressor valve based on CNN and LSTM network[C]//Proceedings of the 2021 International Conference on Power Grid System and Green Energy，2021：252.
[68] LI X，LI J，QU Y，et al.Gear pitting fault diagnosis using integrated CNN and GRU network with both vibration and acoustic emission signals[J].Applied Sciences，2019，9（4）：768.
[69] YANG J，CHEN B，WANG Y N，et al.Crack detection in carbide anvil using acoustic signal and deep learning with particle swarm optimisation[J].Measurement，2021，173：108668.
[70] XIAO D Y，QIN C J，YU H G，et al.Unsupervised machine fault diagnosis for noisy domain adaptation using marginal denoising autoencoder based on acoustic signals[J].Measurement，2021，176：109186.
[71] KOSIOREK A R，SABOUR S，TEH Y W，et al.Stacked capsule autoencoders[C]//Proceedings of the 33rd Annual Conference on Neural Information Processing Systems，2019：32.
[72] CRESWELL A，WHITE T，DUMOULIN V，et al.Generative adversarial networks：an overview[J].IEEE Signal Processing Magazine，2018，35（1）：53-65.
[73] COOPER C，ZHANG J J，GAO R X，et al.Anomaly detection in milling tools using acoustic signals and generative adversarial networks[J].Procedia Manufacturing，2020，48：372-378.
[74] HATANAKA S，NISHI H.Efficient GAN-based unsupervised anomaly sound detection for refrigeration units[C]//Proceedings of the 30th IEEE International Symposium on Industrial Electronics，2021.
[75] TAGAWA Y，MASKELIūNAS R，DAMA?EVI?IUS R.Acoustic anomaly detection of mechanical failures in noisy real-life factory environments[J].Electronics，2021，10（19）：2329.
[76] HE Y，TANG H S，REN Y，et al.A deep multi-signal fusion adversarial model based transfer learning and residual network for axial piston pump fault diagnosis[J].Measurement，2022，192：110889.
[77] ZHUANG F，QI Z，DUAN K，et al.A comprehensive survey on transfer learning[J].Proceedings of the IEEE，2020，109（1）：43-76.
[78] LIU W S，CHEN Z，ZHENG M H.An audio-based fault diagnosis method for quadrotors using convolutional neural network and transfer learning[C]//Proceedings of the 2020 American Control Conference，2020：1367-1372.
[79] HASAN M J，ISLAM M M M，KIM J M.Acoustic spectral imaging and transfer learning for reliable bearing fault diagnosis under variable speed conditions[J].Measurement，2019，138：620-631.
[80] GANIN Y，USTINOVA E，AJAKAN H，et al.Domain-adversarial training of neural networks[J].The Journal of Machine Learning Research，2016，17（1）：2096-2030.
[81] LI Y，SONG Y，JIA L，et al.Intelligent fault diagnosis by fusing domain adversarial training and maximum mean discrepancy via ensemble learning[J].IEEE Transactions on Industrial Informatics，2020，17（4）：2833-2841.
[82] ZHAO C，SHEN W.A domain generalization network combing invariance and specificity towards real-time intelligent fault diagnosis[J].Mechanical Systems and Signal Processing，2022，173：108990.
[83] ZHOU Y，ZHENG H X，HUANG X，et al.Graph neural networks：taxonomy，advances，and trends[J].ACM Transactions on Intelligent Systems and Technology，2022，13（1）：15.
[84] TZIRAKIS P，KUMAR A，DONLEY J.Multi-channel speech enhancement using graph neural networks[C]//Proceedings of the ICASSP 2021，Toronto，Jun 6-11，2021：3415-3419.
[85] ZHANG D，STEWART E，ENTEZAMI M，et al.Intelligent acoustic-based fault diagnosis of roller bearings using a deep graph convolutional network[J].Measurement，2020，156：107585.
[86] ZHANG K Y，CHEN J L，HE S L，et al.Triplet metric driven multi-head GNN augmented with decoupling adversarial learning for intelligent fault diagnosis of machines under varying working condition[J].Journal of Manufacturing Systems，2022，62：1-16.
[87] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Proceedings of the 31st Annual Conference on Neural Information Processing Systems，Long Beach，Dec 4-9，2017：5999-6009.
[88] WU B，CAI W，CHENG F，et al.Simultaneous-fault diagnosis considering time series with a deep learning transformer architecture for air handling units[J].Energy and Buildings，2022，257：111608.
[89] FAN H W，MA N G，ZHANG X H，et al.New intelligent fault diagnosis approach of rolling bearing based on improved vibration gray texture image and vision transformer[J].Proceedings of the Institution of Mechanical Engineers（Part C：Journal of Mechanical Engineering Science），2022.
[90] PEI X，ZHENG X，WU J.Rotating machinery fault diagnosis through a transformer convolution network subjected to transfer learning[J].IEEE Transactions on Instrumentation and Measurement，2021，70：1-11.
[91] ZHOU H，HUANG X，WEN G，et al.Convolution enabled transformer via random contrastive regularization for rotating machinery diagnosis under time-varying working conditions[J].Mechanical Systems and Signal Processing，2022，173：109050.
[92] BUDA M，MAKI A，MAZUROWSKI M A.A systematic study of the class imbalance problem in convolutional neural networks[J].Neural Networks，2018，106：249-259.
[93] ZHANG T C，CHEN J L，LI F D，et al.Intelligent fault diagnosis of machines with small & imbalanced data：a state-of-the-art review and possible extensions[J].ISA Transactions，2022，119：152-171.
[94] LI Q，SHEN C Q，CHEN L，et al.Knowledge mapping-based adversarial domain adaptation：a novel fault diagnosis method with high generalizability under variable working conditions[J].Mechanical Systems and Signal Processing，2021，147：107095.
[95] XIA M，SHAO H，WILLIAMS D，et al.Intelligent fault diagnosis of machinery using digital twin-assisted deep transfer learning[J].Reliability Engineering & System Safety，2021，215：107938.