Lightweight Saliency Object Detection Guided by Deep Feature Aggregation

doi:10.3778/j.issn.1002-8331.2205-0142

Abstract

Abstract: Most of the research on saliency target detection is to pursue performance, while ignoring efficiency, resulting in poor practicability. To this end, this paper proposes an efficient and lightweight network model. Firstly, a feature extraction sub-network（LFRM） is constructed using the idea of feature reuse to fully extract and aggregate the deep feature information of the lightweight feature extraction network, and generate the initial rough saliency prediction map that is used for positioning target guidance of subsequent low-level features. Secondly, according to the differences between feature layers at each stage, a cross-layer interactive aggregation module（CIAM） is constructed to effectively aggregate spatial information and semantic information and reduce redundant information. Finally, an edge refinement module（ERM） is constructed to fully obtain and utilize edge contour information, while adopting a progressive self-guided loss to enhance the dependence of edge information on each other. The final network has only 3.48 ×106 of parameters, and for a 352×352 image, it can reach a running speed of 108 FPS on a single GTX1080Ti graphics card. The test results on five benchmark public datasets show that the model proposed in this paper has comparable or even better performance than the current state-of-the-art SOD methods, with smaller parameters and faster speed.

Key words: saliency object detection, lightweight, feature extraction, edge information

摘要： 目前显著性目标检测的研究大都是追求性能，而忽略了效率，导致实用性较差。为此，提出一个高效且轻量的网络模型，利用特征复用的思想构建了一种特征提取子网络（LFRM）来充分提取与聚合轻量级特征提取网络的深层特征信息，并生成初始粗糙显著预测图，来用于后续低层特征的定位目标指导；针对各阶段特征层之间的差异，构建了一种跨层交互聚合模块（CIAM）来有效进行空间信息与语义信息的聚合，并减少冗余信息；构建了一种边缘细化模块（ERM）来充分获取和利用边缘轮廓信息，同时采用一种渐进式自引导损失来增强边缘信息彼此的依赖性。最终的网络只有3.48×106的参数，且对于352×352的图片，在单张GTX 1080Ti显卡上能够达到108?FPS的运行速度。对五个基准公开数据集的测试结果表明，所提出的模型拥有跟目前最先进的SOD方法相当甚至更好的性能，同时具有更小的参数以及更快的速度。

关键词: 显著性目标检测, 轻量级, 特征提取, 边缘信息

LI Junwen, ZHANG Hongying, HAN Bin. Lightweight Saliency Object Detection Guided by Deep Feature Aggregation[J]. Computer Engineering and Applications, 2023, 59(19): 122-129.

李俊文, 张红英, 韩宾. 深层特征聚合引导的轻量级显著性目标检测[J]. 计算机工程与应用, 2023, 59(19): 122-129.

References

[1] 张鑫，姚庆安，赵健，等.全卷积神经网络图像语义分割方法综述[J].计算机工程与应用，2022，58（8）：45-57.
ZHANG X，YAO Q A，ZHAO J，et al.Image semantic segmentation based on fully convolutional neural network[J].Computer Engineering and Applications，2022，58（8）：45-47.
[2] 钱伍，王国中，李国平.改进YOLOv5的交通灯实时检测鲁棒算法[J].计算机科学与探索，2022，16（1）：231-241.
QIAN W，WANG G Z，LI G P.Improved YOLOv5 traffic light real-time detection robust algorithm[J].Journal of Frontiers of Computer Science and Technology，2022，16（1）：231-241.
[3] 汪雷，黄剑，段涛，等.基于气压肌动图和改进神经模糊推理系统的手势识别研究[J].自动化学报，2022，48（5）：1220-1233.
WANG L，HUANG J，DUAN T，et al.Research on gesture recognition based on pressure-based mechanomyogram and improved neural fuzzy inference system[J].Acta Automatica Sinica，2022，48（5）：1220-1233.
[4] RONNEBERGER O，FISCHER P，BROX T.U-Net：convolutional networks for biomedical image segmentation[C]//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention，2015：234-241.
[5] FENG M，LU H，DING E.Attentive feedback network for boundary-aware salient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：1623-1632.
[6] QIN X，ZHANG Z，HUANG C，et al.U2-Net：going deeper with nested U-structure for salient object detection[J].Pattern Recognition，2020，106：107404.
[7] ZHAO J X，LIU J J，FAN D P，et al.EGNet：edge guidance network for salient object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：8779-8788.
[8] LIU J J，HOU Q，CHENG M M，et al.A simple pooling-based design for real-time salient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：3917-3926.
[9] LIN T Y，DOLLáR P，GIRSHICK R，et al.Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2117-2125.
[10] PANG Y，ZHAO X，ZHANG L，et al.Multi-scale interactive network for salient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：9413-9422.
[11] WEI J，WANG S，HUANG Q.F3Net：fusion，feedback and focus for salient object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：12321-12328.
[12] CHEN Z，XU Q，CONG R，et al.Global context-aware progressive aggregation network for salient object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：10599-10606.
[13] ZHANG M，LIU T，PIAO Y，et al.Auto-MSFNet：search multi-scale fusion network for salient object detection[C]//Proceedings of the 29th ACM International Conference on Multimedia，2021：667-676.
[14] LIU Y，GU Y C，ZHANG X Y，et al.Lightweight salient object detection via hierarchical visual perception learning[J].IEEE Transactions on Cybernetics，2020，51（9）：4439-4449.
[15] ZHOU H，XIE X，LAI J H，et al.Interactive two-stream decoder for accurate and fast saliency detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：9141-9150.
[16] WU Z，SU L，HUANG Q.Cascaded partial decoder for fast and accurate salient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：3907-3916.
[17] LIU Y，ZHANG X Y，BIAN J W，et al.SAMNet：stereoscopically attentive multi-scale network for lightweight salient object detection[J].IEEE Transactions on Image Processing，2021，30：3804-3814.
[18] QIN X B，ZHANG Z C，HUANG C Y，et al.BASNet：boundary-aware salient object detection[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，Jun 16-20，2019：7479-7489.
[19] WANG W，ZHAO S，SHEN J，et al.Salient object detection with pyramid attention and salient edges[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：1448-1457.
[20] HOWARD A，SANDLER M，CHU G，et al.Searching for mobilenetv3[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：1314-1324.
[21] YANG S，LIN W，LIN G，et al.Progressive self-guided loss for salient object detection[J].IEEE Transactions on Image Processing，2021，30：8426-8438.
[22] ZHAO Z，XIA C，XIE C，et al.Complementary trilateral decoder for fast and accurate salient object detection[C]//Proceedings of the 29th ACM International Conference on Multimedia，2021：4967-4975.
[23] LIU J J，HOU Q，CHENG M M.Dynamic feature integration for simultaneous detection of salient object，edge，and skeleton[J].IEEE Transactions on Image Processing，2020，29：8652-8667.
[24] SU J，LI J，ZHANG Y，et al.Selectivity or invariance：Boundary-aware salient object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：3799-3808.