Review of Research on Video-Based Smoke Detection Algorithms

doi:10.3778/j.issn.1002-8331.2203-0506

Abstract

Abstract: In the early stage of fire, the characteristics of smoke are more obvious than that of flame, therefore, early smoke detection is of great significance for preventing fires. For early smoke detection of forest and grassland fires, smoke detectors have high cost, poor detection effect, and cannot provide information such as smoke size, direction, and location. With the development of computer vision, video-based smoke detection methods are more and more widely used in fire detection due to the advantages of low cost, wide coverage, and comprehensive information acquisition. Therefore, video smoke detection algorithms still face huge challenges. In order to study the application effect of deep learning in video smoke detection, it first analyzes the difficulties in smoke detection and the shortcomings of traditional video smoke detection algorithms, and then introduces the application of various target detection algorithms in current deep learning in smoke detection. Then, these smoke detection algorithms are compared, and their advantages and disadvantages are summarized. Finally, the solutions to various difficult smoke detection problems are analyzed, and the next research direction of smoke detection is proposed.

Key words: deep learning, smoke detection, feature extraction

摘要： 火灾早期时，烟雾相较于火焰特征更为明显，因此早期的烟雾检测对于预防火灾具有重大意义。针对森林、草原火灾早期的烟雾检测，烟雾探测器成本高、检测效果较差且不能提供烟雾大小、方向、位置等信息。随着计算机视觉的发展，基于视频的烟雾检测方法以低成本、覆盖面广、信息获取较全面等优势逐渐成为研究热点。但由于森林、草原背景复杂、烟雾本身易变化，因此视频烟雾检测算法仍面临着巨大的挑战。为研究深度学习的视频烟雾检测中的应用效果，分析了烟雾检测存在的难点问题及传统视频烟雾检测算法的不足，介绍了当前深度学习中各类目标检测算法在烟雾检测中的应用，对比了这些烟雾检测算法，总结了其优点和不足，重点分析了各种烟雾检测难点问题的解决方法，并提出烟雾检测的下一步研究方向。

关键词: 深度学习, 烟雾检测, 特征提取

ZHU Mixue, LIU Zhiqiang, ZHANG Xu, LI Wenjing, SU Jiaxin. Review of Research on Video-Based Smoke Detection Algorithms[J]. Computer Engineering and Applications, 2022, 58(14): 16-26.

朱弥雪, 刘志强, 张旭, 李文静, 苏佳新. 林火视频烟雾检测算法综述[J]. 计算机工程与应用, 2022, 58(14): 16-26.

References

[1] 王晓薇，张笑语，孙可，等.基于改进深度信念网络的校园烟雾检测[J].沈阳师范大学学报（自然科学版），2020，38（4）：345-350.
WANG X W，ZHANG X Y，SUN K，et al.Modified deep belief network for campus smoke detection[J].Journal of Shenyang Normal University（Natural Science Edition），2020，38（4）：345-350.
[2] 郝建红，范宗皓，王晖.基于ELU卷积神经网络的视频烟雾检测[J].燕山大学学报，2020，44（4）：397-402.
HAO J H，FAN Z H，WANG H.ELU convolution neural network algorithm for video smoke recognition[J].Journal of Yanshan University，2020，44（4）：397-402.
[3] GU J，WANG Z，KUEN J，et al.Recent advances in convolutional neural networks[J].arXiv：1512.07108，2015.
[4] 刘长春，刘鹏举，季烨云.基于视频区域动态特征的林火烟雾检测技术研究[J].北京林业大学学报，2021，43（1）：10-19.
LIU C C，LIU P J，JI Y Y.Research on forest fire smoke detection technology based on video region dynamic features[J].Journal of Beijing Forestry University，2021，43（1）：10-19.
[5] 王韦刚，王炳蔚，张云伟.TDFF：一种强鲁棒性的烟雾图像检测算法[J].激光与光电子学进展，2021，58（4）：273-280.
WANG W G，WANG B W，ZHANG Y W.TDFF：strong robust algorithm for smoke image detection[J].Laser & Optoelectronics Progress，2021，58（4）：273-280.
[6] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM，2017，60（6）：84-90.
[7] SIMONYAN K，ZISSERMAN A.Very deep convoluteonal networks for large-scale image recognition[EB/OL].（2014-09-04）[2021-09-13].http：//arxiv.org/abs/1409.1556.
[8] HE K M，ZHANG X Y，REN S Q，et al.Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），June 27-30，2016，Las Vegas，NV，USA.New York：IEEE，2016：770-778.
[9] GIRSHICK R，DONAHUE J，DARRELL T，et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2014：580-587.
[10] GIRSHICK R.Fast R-CNN[C]//Proceedings of IEEE International Conference on Computer Vision.Washington：IEEE Computer Society Press，2015：1440-1448.
[11] HE K，ZHANG X，REN S，et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（9）：1904-1916.
[12] REN S Q，HE K M，GIRSHICK R，et al.Faster R-CNN towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（6）：1137-1149.
[13] REDMON J，DIVVALA S，GIRSHICK R，et al.You only look once：unified，real-time object detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition，Las Vegas，Jun 27-30，2016.Washington：IEEE Computer Society，2016：779-788.
[14] REDMON J，FARHADI A.YOLO9000：better，faster stronger[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，Jul 21-26，2017.Washington：IEEE Computer Society，2017：6517-6525.
[15] REDMON J，FARHADI A.Yolov3：an incremental improvement[J].arXiv：1804.02767，2018.
[16] BOCHKOVSKIY A，WANG C Y，LIAO H Y M.YOLOv4：optimal speed and accuracy of object detection[J].arXiv：2004.10934，2020.
[17] WANG C Y，LIAO H Y M，WU Y H，et al.CSPNet：a new backbone that can enhance learning capability of CNN[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops（CVPRW），2020：1571-1580.
[18] LIU S，QI L，QIN H，et al.Path aggregation network for instance segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2018.
[19] LIU W，ANGUELOV D，ERHAN D，et al.SSD：single shot multibox detector[C]//Proceedings of the European Conference on Computer Vision，Amsterdam，October 11-14，2016.Berlin，Heidelberg：Springer，2016：21-37.
[20] FU C Y，LIU W，RANG A，et al.DSSD：deconvolutional single shot detector[J].arXiv：1701.06659，2017.
[21] JENOGN J，PARK H，KWAK N.Enhancement of SSD by concatenating feature maps for object detection[J].arXiv：1705.09587，2017.
[22] LI Z X，ZHOU F Q.FSSD：feature fusion single shot multibox detector[J].arXiv：1712.00960，2017.
[23] 冯路佳，王慧琴，王可，等.基于目标区域的卷积神经网络火灾烟雾识别[J].激光与光电子学进展，2020，57（16）：83-91.
FENG L J，WANG H Q，WANG K，et al.Convolutional neural network fire smoke detection based on target region[J].Laser & Optoelectronics Progress，2020，57（16）：83-91.
[24] 高联欣，魏维，胡泳植，等.基于运动筛选和3D卷积的视频早期烟雾检测[J].计算机工程与应用，2020，56（17）：266-272.
GAO L X，WEI W，HU Y Z，et al.Video early smoke detection based on motion extraction and 3D convolutional neural network[J].Computer Engineering and Applications，2020，56（17）：266-272.
[25] LEE S，LEE Y.Real-time smoke detection research with false positive reduction using spatial and temporal features based on Faster R-CNN[J].Journal of IKEEE，2020，24（4）：1148-1155.
[26] WANG Z，BOVIK A C，SHEIKH H R，et al.Image quality assessment：from error visibility to structural similarity[J].IEEE Transactions on Image Processing，2004，13（4）：600-612.
[27] WANG G，LI J，ZHENG Y，et al.Forest smoke detection based on deep learning and background modeling[C]//IEEE International Conference on Power，Intelligent Computing and Systems（ICPICS），2020：112-116.
[28] 杜立召，徐岩，张为.一种双网融合的分阶段烟雾检测算法[J].西安电子科技大学学报，2020，47（4）：141-148.
DU L Z，XU Y，ZHANG W.Phased smoke detection algorithm using dual network fusion[J].Journal of Xidian University，2020，47（4）：141-148.
[29] YIN H，WEI Y，LIU H，et al.Deep convolutional generative adversarial network and convolutional neural network for smoke detection[J].Complexity，2020，8：1-12.
[30] 李鹏，张炎.基于高斯混合模型和卷积神经网络的视频烟雾检测[J].激光与光电子学进展，2019，56（21）：140-146.
LI P，ZHANG Y.Video smoke detection based on Gaussian mixture model and convolutional neural network[J].Laser & Optoelectronics Progress，2019，56（21）：140-146.
[31] 程淑红，马继勇，张仕军，等.改进的混合高斯与YOLOv2融合烟雾检测算法[J].计量学报，2019，40（5）：798-803.
CHENG S H，MA J Y，ZHANG S J，et al.Smoke detection algorithm combined with improved Gaussian mixture and YOLOv2[J].Acta Metrologica Sinica，2019，40（5）：798-803.
[32] 李澎林，章军伟，李伟.基于光流改进与YOLOv3的烟雾检测方法[J].浙江工业大学学报，2021，49（1）：9-15.
LI P L，ZHANG J W，LI W.Smoke detection method based on optical flow improvement and YOLOv3[J].Journal of Zhejiang University of Technology，2021，49（1）：9-15.
[33] 叶寒雨，李传昌，刘淼，等.基于稠密光流和目标检测的烟雾检测算法[J/OL].电光与控制：1-7[2022-03-18].http：//kns.cnki.net/kcms/detail/41.1227.tn.20220314.1205.006.html.
YE H Y，LI C C，LIU M，et al.Smoke detection method based on dense optical flow and target detection[J/OL].Electronics Optics & Control：1-7[2022-03-18].http：//kns.cnki.net/kcms/detail/41.1227.tn.20220314.1205.006.html.
[34] SHENG D，DENG J，XIANG J.Automatic smoke detection based on SLIC-DBSCAN enhanced convolutional neural network[J].IEEE Access，2021，9：63933-63942.
[35] PUNDIR A S，RAMAN B.Dual deep learning model for image based smoke detection[J].Fire Technol，2019，55：2419-2442.
[36] BAI X，WANG Z.Research on forest fire detection technology based on deep learning[C]//International Conference on Computer Network，Electronic and Automation（ICCNEA），2021：85-90.
[37] 谢书翰，张文柱，程鹏，等.嵌入通道注意力的YOLOv4火灾烟雾检测模型[J].液晶与显示，2021，36（10）：1445-1453.
XIE S H，ZHANG W Z，CHENG P，et al.Fire smoke detection model based on YOLOv4 with channel attention[J].Chinese Journal of Liquid Crystals and Displays，2021，36（10）：1445-1453.
[38] HU J，SHEN L，ALBANIE S，et al.Squeeze-and-excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，42（8）：2011-2023.
[39] 汪梓艺，苏育挺，刘艳艳，等.一种改进DeeplabV3网络的烟雾分割算法[J].西安电子科技大学学报，2019，46（6）：52-59.
WANG Z Y，SU Y T，LIU Y Y，et al.Algorithm for segmentation of smoke using the improved DeeplabV3 network[J].Journal of Xidian University，2019，46（6）：52-59.
[40] CAI W，WANG C，HUANG H，et al.A real-time smoke detection model based on YOLO-SMOKE algorithm[C]//Cross Strait Radio Science & Wireless Technology Conference（CSRSWTC），2020：1-33.
[41] WANG Q，WU B，ZHU P，et al.ECA-Net：efficient channel attention for deep convolutional neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2020：11531-11539.
[42] ZHANG D，CAO Y，ZHANG G，et al.An attention convolutional neural network for forest fire smoke recognition[C]//International Conference on Systems and Informatics（ICSAI），2019：1207-1211.
[43] WOO S，PARK J，LEE J，et al.Cbam：convolutional block attention module[J].arXiv：1807.06521，2018.
[44] XU G，ZHANG Y，ZHANG Q，et al.Video smoke detection based on deep saliency network[J].Fire Safety Journal，2019，105：277-285.
[45] 杨龙箴，袁非牛，杨寿渊，等.连续图卷积视频烟雾检测模型[J].中国图象图形学，2019，24（10）：1658-1669.
YANG L Z，YUAN F N，YANG S Y，et al.Continuous graph convolutional model for video smoke detection[J].Journal of Image and Graphics，2019，24（10）：1658-1669．
[46] GU K，XIA Z，QIAO J，et al.Deep dual-channel neural network for image-based smoke detection[J].IEEE Transactions on Multimedia，2020，22（2）：311-323.
[47] LIU T，CHENG J，DU X，et al.Video smoke detection method based on change-cumulative image and fusion deep network[J].Sensors，2019，19（23）：5060.
[48] 王洋，程江华，刘通，等.一种多网络模型融合的烟雾检测方法[J].计算机工程与科学，2019，41（10）：1771-1776.
WANG Y，CHENG J H，LIU T，et al.A smoke detection method based on fusing multiple network models[J].Computer Engineering & Science，2019，41（10）：1771-1776.
[49] 刘丽娟，陈松楠.一种基于改进SSD的烟雾实时检测模型[J].信阳师范学院学报（自然科学版），2020，33（2）：305-311.
LIU L J，CHEN S N.Real-time smoke detection model based on improved SSD[J].Journal of Xinyang Normal University（Natural Science Edition），2020，33（2）：305-311.
[50] 朱傥，杨忠，周国兴，等.一种轻量化网络的火焰烟雾检测算法[J].应用科技，2022，49（2）：1-7.
ZHU T，YANG Z，ZHOU G X，et al.A flame and smoke detection algorithm for lightweight networks[J].Applied Science and Technology，2022，49（2）：1-7.
[51] ZHAO Y，ZHANG H，ZHANG X，et al.Fire smoke detection based on target-awareness and depthwise convolutions[J].Multimedia Tools and Applications，2021，80（18）：27407-27421.
[52] HUO Y，ZHANG Q，JIA Y，et al.A deep separable convolutional neural network for multiscale image-based smoke detection[J].Fire Technology，2022，58：1445-1468.
[53] YIN Z，WAN B，YUAN F，et al.A deep normalization and convolutional neural network for image smoke detection[J].IEEE Access，2017，5：18429-18438.
[54] LIN G H，ZHANG Y M，XU G，et al.Smoke detection on video sequences using 3D convolutional neural networks[J].Fire Technology，2019：1-21.
[55] 谢宏，陈祎婧，袁小芳，等.时空双路3D残差卷积网络的视频烟雾检测[J].计算机工程与应用，2020，56（18）：143-149.
XIE H，CHEN Y J，YUAN X F，et al.Video smoke detection using spatiotemporal dual path 3D residual convolutional network[J].Computer Engineering and Applications，2020，56（18）：143-149.
[56] 吴凡，王慧琴，王可.时空域深度学习火灾烟雾检测[J].液晶与显示，2021，36（8）：1186-1195.
WU F，WANG H Q，WANG K.Spatio-temporal deep learning fire smoke detection[J].Chinese Journal of Liquid Crystals and Displays，2021，36（8）：1186-1195.
[57] PENG Y，WANG Y.Real-time forest smoke detection using hand-designed features and deep learning[J].Computers and Electronics in Agriculture，2019，167：105029.
[58] 刘通，程江华，华宏虎，等.结合YdUaVa颜色模型和改进MobileNetV3的视频烟雾检测方法[J].国防科技大学学报，2021，43（5）：80-85.
LIU T，CHENG J H，HUA H H，et al.Video smoke detection method combining YdUaVa color model and improved MobileNetV3[J].Journal of National University of Defense Technology，2021，43（5）：80-85.
[59] 吉森荣.基于改进Tiny-YOLOv3的烟雾检测算法[J].科学技术创新，2021（4）：95-96.
JI S R.Smoke detection algorithm based on improved Tiny-YOLOv3[J].Science and Technology Innovation，2021（4）：95-96.
[60] 陈朝晖，陈之宇.基于人工智能的轻量级模型对烟雾检测研究及应用[J].消防科学与技术，2020，39（12）：1747-1750.
CHEN Z H，CHEN Z Y.Research and application of lightweight model based on artificial intelligence for smoke detection[J].Fire Science and Technology，2020，39（12）：1747-1750.
[61] PAN J，OU X，XU L，et al.A collaborative region detection and grading framework for forest fire smoke using weakly supervised fine segmentation and lightweight Faster-RCNN[J].Forests，2021，12（6）：768.
[62] XIE C，TAO H.Generating realistic smoke images with controllable smoke components[J].IEEE Access，2020，8：201418-201427.
[63] LI T，ZHU H，HU C，et al.An attention-based prototypical network for forest fire smoke few-shot detection[J].Journal of Forestry Research，2022.
[64] WU Y，CHEN M，WO Y，et al.Video smoke detection base on dense optical flow and convolutional neural network[J].Multimedia Tools and Applications，2021，80（28）：35887-35901.
[65] 王文朋，毛文涛，何建樑，等.基于深度迁移学习的烟雾识别方法[J].计算机应用，2017，37（11）：3176-3181.
WANG W P，MAO W T，HE J L，et al.Smoke recognition based on deep transfer learning[J].Journal of Computer Applications，2017，37（11）：3176-3181.
[66] 张倩，周平平，王公堂，等.基于合成图像的Faster R-CNN森林火灾烟雾检测[J].山东师范大学学报（自然科学版），2019，34（2）：180-185.
ZHANG Q，ZHOU P P，WANG G T，et al.Faster R-CNN forest fire smoke detetion based on synthetic images[J].Journal of Shandong Normal University（Natural Science），2019，34（2）：180-185.
[67] 袁梅，全太锋，黄俊，等.基于卷积神经网络的烟雾检测[J].重庆邮电大学学报（自然科学版），2020，32（4）：620-629.
YUAN M，QUAN T F，HUANG J，et al.Smoke detection based on convolutional neural network[J].Journal of Chongqing University of Posts and Telecommunications（Natural Science Edition），2020，32（4）：620-629.