Saliency Detection from Complex Background with Attention-Based Boundary-Aware Pyramid Pooling Network

doi:10.3778/j.issn.1002-8331.2206-0392

Abstract

Abstract: Deep convolution networks have been widely adopted in saliency detection in recent years and achieved state-of-the-art performance. A core issue for previous methods is the limited capability of detecting objects that are overwhelmed by complex background. Aiming to address this problem, a novel attention-based boundary-aware pyramid pooling network is proposed, namely ABAPNet, which consists of feature aggregation module, pyramid pooling module and cascading dual attention module. The proposed ABAPNet method transits high-level information to shallow layers to keep transferring important features between layers and obtains more semantic features. Hybrid loss by fusing binary cross entropy, structural similarity and intersection-over-union is employed to reduce the gradient of mixed loss of network output features, which guides the network to better segment boundaries from complex background. Experimental results on 5 public datasets show that the proposed ABAPNet outperforms the competitors.

Key words: saliency object detection, attention mechanism, deep learning, pyramid pooling, boundary-aware

摘要： 近年来，深度卷积神经网络在显著性目标检测中得到广泛关注和研究，并取得了重要进展。但当显著性目标处于复杂背景中时，当前算法的性能仍有待提高。提出一种利用边界感知和注意力机制的金字塔池化网络（attention-based boundary-aware pyramid pooling network，ABAPNet），用于复杂场景下的显著性目标检测。ABAPNet通过引入级联式通道注意力和空间注意力机制，采用特征金字塔网络架构获取多层次的语义特征，以高层特征信息来辅助浅层特征；再通过融合二进制交叉熵、结构相似性和联合交集的混合损失函数，可增强获取重要语义特征并且关注目标边界特征，从而引导网络从复杂背景中更好地检测目标。在5个公开数据集上的实验表明，ABAPNet在多个评价指标上均优于比较算法，达到最优性能。

关键词: 显著性目标检测, 注意力机制, 深度学习, 金字塔池化, 边界感知

FANG Jinsheng, TAO Yuhao, ZHU Gupei, CHEN Yanyou. Saliency Detection from Complex Background with Attention-Based Boundary-Aware Pyramid Pooling Network[J]. Computer Engineering and Applications, 2023, 59(22): 259-267.

方金生, 陶余昊, 朱古沛, 陈彦佑. 复杂场景下显著性目标检测注意力金字塔网络[J]. 计算机工程与应用, 2023, 59(22): 259-267.

References

[1] ITTI L，KOCH C，NIEBUR E.A model of saliency-based visual attention for rapid scene analysis[J].IEEE Transaction on Pattern Analysis and Machine Intelligence，1998，20（11）：1254-1259.
[2] 左保川，张晴.采用特征引导机制的显著性检测网络[J].计算机工程与应用，2021，57（14）：201-208.
ZUO B C，ZHANG Q.Feature guidance mechanism for saliency detection network[J].Computer Engineering and Applications，2021，57（14）：201-208.
[3] 史彩娟，张卫明，陈厚儒，等.基于深度学习的显著性目标检测综述[J].计算机科学与探索，2021，15（2）：219-232.
SHI C J，ZHANG W M，CHEN H R，et al.Survey of salient object detection based on deep learning[J].Journal of Frontiers of Computer Science and Technology，2021，57（2）：214-219.
[4] RUI Z，OUYANG W，LI H，et al.Saliency detection by multi-context deep learning[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition，Boston，2015：1265-1274.
[5] LIN T Y，DOLLAR P，GIRSHICK R，et al.Feature pyramid networks for object detection[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，2017：936-944.
[6] LIU J J，HOU Q，CHENG M M，et al.A simple pooling-based design for real-time salient object detection[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：3912-3921.
[7] QIN X，ZHANG Z，HUANG C，et al.BASNet：boundary-aware salient object detection[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，2019：7471-7481.
[8] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Proceedings of the 31st Annual Conference on Neural Information Processing Systems，Long Beach，2017：6000-6010.
[9] ZHU X，SU W，LU L，et al.Deformable DETR：deformable transformers for end-to-end object detection[J].arXiv：201004159，2020.
[10] 罗会兰，袁璞，童康.基于深度学习的显著性目标检测方法综述[J].电子学报，2021，49（7）：1417-1427.
LUO H L，YUAN P，TONG K.Review of the methods for salient object detection based on deep learning[J].Acta Electronica Sinica，2021，49（7）：1417-1427.
[11] LI G，YU Y.Visual sliency detection based on multiscale deep CNN features[J].IEEE Transactions on Image Processing，2016，25（11）：5012-5024.
[12] WANG L，LU H，XIANG R，et al.Deep networks for saliency detection via local estimation and global search[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition，Boston，2015：3183-3192.
[13] LIU N，HAN J.DHSNet：deep hierarchical saliency network for salient object detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition，Seattle，2016：678-686.
[14] HOU Q，CHENG M M，HU X，et al.Deeply supervised salient object detection with short connections[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，2017：5300-5309.
[15] WANG L，LU H，WANG Y，et al.Learning to detect salient objects with image-level supervision[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，2017：3796-3805.
[16] DENG Z，HU X，ZHU L，et al.R3Net：recurrent residual refinement network for saliency detection[C]//Proceedings of the 2018 International Joint Conference on Artificial Intelligence，Stockholm，2018：684-690.
[17] ZHANG P P，WANG D，LU H C，et al.Amulet：aggregating multi-level convolutional features for salient object detection[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision，Venice，2017：202-211.
[18] LIU N，HAN J，YANG M H.PiCANet：learning pixel-wise contextual attention for saliency detection[J].IEEE Transactions on Image Processing，2017，29：6438-6451.
[19] 何伟，潘晨.注意力引导网络的显著性目标检测[J].中国图象图形学报，2022，27（4）：1176-1190.
HE W，PAN C.The salient object detection based on attention-guided network[J].Chinese Journal of Image and Graphics，2022，27（4）：1176-1190.
[20] 王正文，宋慧慧，樊佳庆.基于语义引导特征聚合的显著性目标检测网络[J].自动化学报，2021，48：1001-1010.
WANG Z W，SONG H H，FAN J Q.Semantic guided feature aggregation network for salient object detection[J].Acta Automatica Sinica，2021，48：1001-1010.
[21] SIMONYAN K，ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv：1409.
1556，2014.
[22] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[23] KRIZHEVSKY A，SUTSKEVER I，HINTON G.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM，2012，60（6）：84-90.
[24] WANG T，BORJI A，ZHANG L，et al.A stagewise refinement model for detecting salient objects in images[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision，Venice，2017：4039-4048.
[25] YAN Q，XU L，SHI J，et al.Hierarchical saliency detection[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition，Portland，2013：1155-1162.
[26] LI Y，HOU X D，KOCH C，et al.The secrets of salient object segmentation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition，Columbus，2014：280-287.
[27] YANG C，ZHANG L，LU H，et al.Saliency detection via graph-based manifold ranking[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition，Portland，2013：3166-3173.
[28] WILLMOTT C，MATSUURA K.Advantages of the mean absolute error（MAE） over the root mean square error （RMSE） in assessing average model performance[J].Climate Research，2005，30（1）：79-82.
[29] HAND D，CHRISTEN P.A note on using the F-measure for evaluating record linkage algorithms[J].Statistics & Computing，2017，28（3）：539-547.
[30] ZHANG X，WANG T，QI J，et al.Progressive attention guided recurrent network for salient object detection[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake City，2018：714-722.
[31] WANG T，ZHANG L，WANG S，et al.Detect globally，refine locally：anovel approach to saliency detection[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake City，2018：3127-3135.
[32] QIN X，ZHANG Z，HUANG C，et al.U2-Net：going deeper with nested U-structure for salient object detection[J].Pattern Recognition，2020，106：107404.
[33] WU Z，SU L，HUANG Q M，et al.Cascaded partial decoder for fast and accurate salient object detection[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，2019：3902-3911.