双分支GAN与注意力机制的火灾隐患检测算法

doi:10.3778/j.issn.1002-8331.2304-0288

摘要/Abstract

摘要： 针对传统火灾报警在夜间等极端天气下效果不佳，受限于复杂环境等问题，提出一种基于红外与可见光图像融合的火灾预警算法。在生成对抗网络(GAN)中设计并提出双分支注意力结构。其中一条分支通过密集残差子网提取更多鲁棒的特征信息，另一条分支通过注意力子网(efficient coordinate channel attention group,ECCAG)弥补空间信息的缺失，以最大限度获取更多高频细节特征，设计并提出了一种调节损失作为损失函数，通过改进GAN算法得到融合图像，根据提出的火灾预警算法判断是否存在火灾隐患。实验结果表明：改进GAN算法得到的融合数据集目标检测的平均准确率为96.19%，相较于单一红外数据集与原始GAN算法数据集的目标检测平均准确率分别提高了11.09个百分点与6.2个百分点，在公开数据集TNO与LLVIP数据集上测试火灾患检测准确率为97.45%。结果表明，火灾预警算法可以在未发生火灾时及时预警，针对不同场景都可得到显著的检测效果。

关键词: 生成对抗网络, 图像融合, 早期火灾预警, 双分支结构, 注意力机制

Abstract: Aiming at the problems of traditional fire alarm can not be warned before the occurrence of fire, the effect is not good in extreme weather such as night, and it is limited by complex environment, a fire alarm algorithm based on infrared and visible light images fusion is proposed. A two-branch attention structure is designed and proposed in a generative adversarial network (GAN). One branch extracts more robust feature information through the dense residual network, and the other branch makes up for the lack of spatial information through the efficient coordinate channel attention group (ECCAG) to maximize the acquisition of more high-frequency detail features, and designs and proposes a regulation loss as a loss function, and obtains the fusion image by improving the GAN algorithm. Finally, according to the proposed fire warning algorithm, whether there is a fire hazard is judged. The experimental results show that, the average accuracy of object detection in the fusion dataset obtained by the improved GAN algorithm is 96.19%, which is improved by 11.09?percentage points and 6.2?percentage points compared with the infrared dataset and the original GAN algorithm dataset, respectively, and the accuracy of flame hazard detection on the TNO and LLVIP datasets of the public dataset is 97.45%. The results show that the fire warning algorithm can warn in time when no fire occurs, and can obtain significant detection effects for different scenarios.

Key words: generative adversarial network, image fusion, early fire warning, two-branched structure, attention mechanism

李牧, 何金诚, 杨恒. 双分支GAN与注意力机制的火灾隐患检测算法[J]. 计算机工程与应用, 2024, 60(14): 228-239.

LI Mu, HE Jincheng, YANG Heng. Fire Hazard Detection Algorithm with Dual-Branch GAN and Attention Mechanism[J]. Computer Engineering and Applications, 2024, 60(14): 228-239.

参考文献

[1] 罗小权, 潘善良. 改进YOLOV3的火灾检测方法[J]. 计算机工程与应用, 2020, 56(17): 187-196.
LUO X Q, PAN S L. Improved YOLOV3 fire detection method[J]. Computer Engineering and Applications, 2020, 56(17): 187-196.
[2] 喻丽春, 刘金清. 基于改进Mask R-CNN的火焰图像识别算法[J]. 计算机工程与应用, 2020, 56(21): 194-198.
YU L C, LIU J Q. Fire image recognition algorithm based on improved Mask R-CNN[J]. Computer Engineering and Applications, 2020, 56(21): 194-198.
[3] 张正贺, 黄贞, 陈汝婷, 等. 基于投影式背景纹影技术的火焰温度场测量仪设计[J]. 激光与光电子学进展, 2019, 56(5): 235-242.
ZHANG Z H, HUANG Z, CHEN R T, et al. Design of flame temperature measurement instrument based on projective background-oriented schlieren technique[J]. Laser & Optoelectronics Progress, 2019, 56(5): 235-242.
[4] CIPRIáN-SáNCHEZ J F, OCHOA-RUIZ G, GONZALEZ-MENDOZA M, et al. FIRe-GAN: a novel deep learning-based infrared-visible fusion method for wildfire imagery[J]. Neural Computing and Applications, 2021, 35: 18201-18213.
[5] CHEN X W, HOPKINS B, WANG H, et al. Wildland fire detection and monitoring using a drone-collected RGB/IR image dataset[J]. IEEE Access, 2022, 10: 121301-121317.
[6] KACKER T, PERRUSQUIA A, GUO W S. Multi-spectral fusion using generative adversarial networks for UAV detection of wild fires[C]//Proceedings of the 2023 International Conference on Artificial Intelligence in Information and Communication, 2023: 182-187.
[7] TLIG M, BOUCHOUICHA M, SAYADI M, et al. Infrared-visible images’ fusion techniques for forest fire monitoring[C]//Proceedings of the 2022 6th International Conference on Advanced Technologies for Signal and Image Processing, 2022: 1-6.
[8] NEMALIDINNE S M, GUPTA D. Nonsubsampled contourlet domain visible and infrared image fusion framework for fire detection using pulse coupled neural network and spatial fuzzy clustering[J]. Fire Safety Journal, 2018, 101: 84-101.
[9] 王继霄, 李阳, 王家宝, 等. 基于SqueezeNet的轻量级图像融合方法[J]. 计算机应用, 2020, 40(3): 837-841.
WANG J X, LI Y, WANG J B, et al. Lightweight image fusion method based on SqueezeNet[J]. Computer Applications, 2020, 40(3): 837-841.
[10] 侯春萍, 王霄聪, 夏晗, 等. 基于双通路生成对抗网络的红外与可见光图像融合方法[J]. 激光与光电子学进展, 2021, 58(14): 318-327.
HOU C P, WANG X C, XIA H, et al. Infrared and visible image fusion method based on dual-channel generative adversarial network[J]. Laser & Optoelectronics Progress, 2021, 58(14): 318-327.
[11] 尹来国, 孙仁诚, 邵峰晶, 等. 多模式生成对抗网络[J]. 计算机应用研究, 2022, 39(6): 1689-1693.
YIN L G, SUN R C, SHAO F J, et al. Multi-mode generative adversarial network[J]. Application Research of Computers, 2022, 39(6): 1689-1693.
[12] 陈强, 朱立新, 夏德深. 结合Canny算子的图像二值化[J]. 计算机辅助设计与图形学学报, 2005, 17(6): 1302-1306.
CHEN Q, ZHU L X, XIA D S. Image binarization based on canny’s operator[J]. Journal of Compute-Aided Design and Graphics, 2005, 17(6): 1302-1306.
[13] 曲海成, 佟畅, 刘万军. 注意力与多尺度融合的图像阴影去除算法[J]. 计算机工程与应用, 2022, 58(16): 234-241.
QU H C, TONG C, LIU W J. Image shadow removal algorithm based on attention and multi-scale fusion[J]. Computer Engineering and Applications, 2022, 58(16): 234-241.
[14] 李晨瑄, 钱坤, 胥辉旗. 基于深浅层特征融合的舰船要害关键点检测算法[J]. 系统工程与电子技术, 2021, 43(11): 3239-3249.
LI C X, QIAN K, XU H Q. Detection algorithm of key points of ships based on deep and shallow feature fusion[J]. Systems Engineering and Electronics, 2021, 43(11): 3239-3249.
[15] CHAI X, WANG Y, CHEN X, et al. TPE-GAN: thumbnail preserving encryption based on GAN with key[J]. IEEE Signal Processing Letters, 2022, 29: 972-976.
[16] GUO Y, LI H, ZHUANG P. Underwater image enhancement using a multiscale dense generative adversarial network[J]. IEEE Journal of Oceanic Engineering, 2020, 45(3): 862-870.
[17] 沈怀艳, 吴云. 基于MSFA-Net的肝脏CT图像分割方法[J]. 计算机科学与探索, 2023, 17(3): 646-656.
WU H Y, WU Y. Liver CT image segmentation method based on MSFA-Net[J]. Journal of Frontiers of Computer Science & Technology, 2023, 17(3): 646-656.
[18] 马永康, 刘华, 凌成星, 等. 基于改进YOLOv5的红树林单木目标检测研究[J]. 激光与光电子学进展, 2022, 59(18): 1828003.
MA Y K, LIU H, LIING C X, et al. Object detection of individual mangrove based on improved YOLOv5[J]. Laser & Optoelectronics Progress, 2022, 59(18): 1828003.
[19] LIU K H, YE Z H, GUO H Y， et al. FISS GAN: a generative adversarial network for foggy image semantic segmentation[J]. IEEE/CAA Journal of Automatica Sinica, 2021, 8(8): 1428-1439.
[20] 夏鸿斌, 肖奕飞, 刘渊. 融合自注意力机制的长文本生成对抗网络模型[J]. 计算机科学与探索, 2022, 16(7): 1603-1610.
XIAO H B, XIAO Y F, LIU Y. Long text generation adversarial network model incorporating self-attention mechanism[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1603-1610.
[21] 程艳, 蔡壮, 吴刚, 等. 结合自注意力特征过滤分类器和双分支GAN的面部表情识别[J]. 模式识别与人工智能, 2022, 35(3): 243-253.
CHENG Y, CAI Z, WU G, et al. Facial expression recognition based on self-attention feature filter classifier and two-branch GAN[J]. Pattern Recognition and Artificial Intelligence, 2022, 35(3): 243-253.
[22] 苏晏园, 范广宇, 龚海梅, 等. InGaAs近红外人脸图像检测超轻量算法研究[J]. 红外与激光工程, 2022, 51(10): 343-352.
SU Y Y, FAN G Y, GONG H M, et al. Research on ultra-lightweight algorithm for InGaAs near-infrared face image detection[J]. Infrared and Laser Engineering, 2022, 51(10): 343-352.
[23] 刘朋远, 田瑞, 周媛奉, 等. 基于自适应混合阈值的智能电表图像二值化[J]. 计算机应用与软件, 2023, 40(1): 210-215.
LIU P Y, TIAN R, ZHOU Y F, et al. Smart meter image binarization based on adaptive mixing threshold[J]. Computer Applications and Software, 2023, 40(1): 210-215.
[24] YANG Z G, CHEN Y P, LE Z L, et al. GANFuse: a novel multi-exposure image fusion method based on generative adversarial networks[J]. Neural Computing & Applications, 2021, 33(11): 6133-6145.
[25] 刘锃亮, 张宇, 吕恒毅. 基于生成对抗网络的可见光与红外图像融合[J]. 无线电工程, 2022, 52(4): 555-561.
LIU Z L, ZHANG Y, LV H Y. Fusion of visible and infrared images based on generative adversarial networks[J]. Radio Engineering, 2022, 52(4): 555-561.
[26] 韩正汀, 路文, 杨舒羽, 等. 基于导向滤波优化的自然图像去雾新方法[J]. 计算机科学与探索, 2015, 9(10): 1256-1262.
HAN Z T, LU W, YANG S Y, et al. Improved natural image dehazing algorithm based on guided filtering[J]. Journal of Frontiers of Computer Science and Technology, 2015, 9(10): 1256-1262.
[27] LIU J Y, FAN X, HUANG Z B, et al. Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 5792-5801.
[28] WANG T, HE J B, XIONG S H, et al. Visual perception enhancement for HEVC compressed video using a generative adversarial network[C]//Proceedings of the 2020 International Conference on UK-China Emerging Technologies, 2020: 1-4.