Salient Object Detection for Enhanced Feature Pyramid Structure

doi:10.3778/j.issn.1002-8331.2011-0111

Abstract

Abstract: Salient object detection plays an important role in the field of computer vision and is a hot research spot. However, there are some problems in its algorithm, such as the single loss function often ignores the influence of multi-dimensional features; the fusion of feature images usually depends on simple addition of pixels, it cannot effectively highlight the region of salient object in the image. In order to overcome above problems, this paper captures the image details by combining structural similarity, intersection over union and cross entropy loss functions. Fusion of feature maps is with multiplication operation that can make the model more sensitive to the salient region. Residual feature map enhancement module is used to build the higher level feature map to strengthen its semantic information. Finally, the feature pyramid structure is used to fuse the information of different scales to complete the coding and decoding module. Experimental comparison on five benchmarks show that the performance of the presented method is better than the popular methods and can detect salient object effectively.

Key words: salient object detection, feature pyramid network, enhanced residual fusion

摘要： 显著目标检测是计算机视觉的研究热点。显著目标检测算法存在一些问题，如：算法常采用单一损失函数，缺乏对多维特征损失的考虑，可能带来局限性；最高层特征图来源单一；特征图融合常使用对应像素相加，不能有效突出图像中感兴趣区域。针对上述问题，结合结构性相似、交并比和交叉熵三种损失函数来捕捉图像细节，采用对应像素相乘操作融合特征图，令模型对显著区域更加敏感；通过残差特征图增强模块逆向构建更高层特征图强化其语义信息；采用特征金字塔结构融合不同尺度信息，完成编码解码模块。在5个数据集的对比实验表明该方法性能超过主流算法，能实现有效的显著目标检测。

关键词: 显著目标检测, 特征金字塔网络, 残差特征图增强

LIU Jianfeng, PAN Chen. Salient Object Detection for Enhanced Feature Pyramid Structure[J]. Computer Engineering and Applications, 2022, 58(12): 226-233.

刘剑峰, 潘晨. 增强特征金字塔结构的显著目标检测算法[J]. 计算机工程与应用, 2022, 58(12): 226-233.

References

[1] GAO Y，WANG M，TAO D，et al.3-D object retrieval and recognition with hypergraph analysis[J].IEEE Transactions on Image Processing，2012，21（9）：4290-4303.
[2] CRAYE C，FILLIAT D，GOUDOU J F.Environment exploration for object-based visual saliency learning[C]//2016 IEEE International Conference on Robotics and Automation（ICRA），2016：2303-2309.
[3] MAHADEVAN V，VASCONCELOS N.Saliency-based discriminant tracking[C]//IEEE Conference on Computer Vision and Pattern Recognition，2009：1007-1013.
[4] REN Z，GAO S，CHIA L，et al.Region-based saliency detection and its application in object recognition[J].IEEE Transactions on Circuits and Systems for Video Technology，2014，24（5）：769-779.
[5] LIN T Y，DOLLáR P，GIRSHICK R，et al.Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2117-2125.
[6] HOU Q，CHENG M M，HU X，et al.Deeply supervised salient object detection with short connections[C]//IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2017：5300-5309.
[7] ZHANG P，LIU W，LU H，et al.Salient object detection by lossless feature reflection[C]//Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence（IJCAI），2018：1149-1155.
[8] WANG L，WANG L，LU H，et al.Saliency detection with recurrent fully convolutional networks[C]//European Conference on Computer Vision，2016：825-841.
[9] ZHANG X，WANG T，QI J，et al.Progressive attention guided recurrent network for salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：714-722.
[10] ZHANG P，WANG D，LU H，et al.Amulet：aggregating multi-level convolutional features for salient object detection[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：202-211.
[11] GADOSEY P K，LI Y，AGYEKUM E A，et al.SD-UNet：stripping down U-Net for segmentation of biomedical images on platforms with low computational budgets[J].Diagnostics，2020，10（2）：110.
[12] WANG Z，SIMONCELLI E P，BOVIK A C.Multiscale structural similarity for image quality assessment[C]//The Thrity-Seventh Asilomar Conference on Signals，Systems & Computers，2003：1398-1402.
[13] MáTTYUS G，LUO W，URTASUN R.Deeproadmapper：extracting road topology from aerial images[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：3438-3446.
[14] DE BOER P T，KROESE D P，MANNOR S，et al.A tutorial on the cross-entropy method[J].Annals of Operations Research，2005，134（1）：19-67.
[15] QIN X，ZHANG Z，HUANG C，et al.Basnet：boundary-aware salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：7479-7489.
[16] ZHANG L，DAI J，LU H，et al.A bi-directional message passing model for salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：1741-1750.
[17] HOU Q，CHENG M M，HU X，et al.Deeply supervised salient object detection with short connections[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3203-3212.
[18] ZHANG X，WANG T，QI J，et al.Progressive attention guided recurrent network for salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：714-722.
[19] CHEN S，TAN X，WANG B，et al.Reverse attention for salient object detection[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：234-250.
[20] LI X，YANG F，CHENG H，et al.Contour knowledge transfer for salient object detection[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：355-370.
[21] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[22] JOINER W M，CAVANAUGH J，WURTZ R H，et al.Visual responses in FEF，unlike V1，primarily reflect when the visual context renders a receptive field salient[J].Journal of Neuroscience，2017，37（41）：9871-9879.
[23] ZHAO H，SHI J，QI X，et al.Pyramid scene parsing network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2881-2890.
[24] WANG L，LU H，WANG Y，et al.Learning to detect salient objects with image-level supervision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：136-145.
[25] LI G，YU Y.Visual saliency based on multiscale deep features[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：5455-5463.
[26] LI Y，HOU X，KOCH C，et al.The secrets of salient object segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2014：280-287.
[27] MOVAHEDI V，ELDER J H.Design and perceptual validation of performance measures for salient object segmentation[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops，2010：49-56.
[28] YANG C，ZHANG L，LU H，et al.Saliency detection via graph-based manifold ranking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2013：3166-3173.
[29] EHRIG M，EUZENAT J.Relaxed precision and recall for ontology matching[C]//Workshop on Integrating Ontologies，2005：25-32.
[30] PERAZZI F，KRAHENBUHL P，PRITCH Y，et al.Saliency filters：contrast based filtering for salient region detection[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition，2012：733-740.