Curb Segmentation Using Dual Branch and Feature Fusion Network

doi:10.3778/j.issn.1002-8331.2202-0208

Abstract

Abstract: Curb detection is an important goal of intelligent vehicle environmental perception. This paper uses semantic segmentation to detect curbobject. Aiming at the problem that the semantic segmentation network cannot balance shallow features and deep features, a real-time curb segmentation network with dual-branch feature fusion is designed. The main branch of the network uses the residual structure module for downsampling, and reverts to the original resolution when the feature map resolution is 1/16 of the input resolution. Multiple modules are used to fuse shallow spatial features and high-level semantic features. The SDFE（spatial detail feature extraction）module is used to make up for the loss of geometric features. The joint feature pyramid（JFP）module is used to have strong semantics in multiple stages of the network. The multi-scale features of information are used in combination, the spatial feature attention mechanism（feature attention, FA）is designed in the branch, and 4 convolution normalization is used to enhance the extraction of spatial detail features based on the deal with the attention. FFM（feature fusion module） is designed. The module fuses high-level semantic features with shallow features. The performance of the network is evaluated. The network test mIoU is 79.65% and the FPS is 59.6. The experiment is carried out on the road, and the segmentation effect is fast and good.

Key words: curb segmentation, dual branch, features fusion

摘要： 路沿检测是智能车辆环境感知的重要目标，使用语义分割的方法对路沿目标进行检测。针对语义分割网络不能平衡浅层特征和深度特征的问题，设计了一种具有双支路特征融合的实时路沿分割网络。网络主支使用残差结构模块进行下采样，在特征图分辨率为输入分辨率的1/16时恢复至原来分辨率。采用多个模块来融合浅层空间特征与高级语义特征：使用SDFE（spatial detail feature extraction）模块弥补几何特征的丢失；使用联合特征金字塔（joint feature pyramid，JFP）模块将网络多个阶段具有强语义信息的多尺度特征结合使用；支路中设计了特征注意力机制（feature attention，FA）模块，使用4个卷积归一化，通过注意力模块处理，用来增强空间细节特征的提取；设计了FFM（feature fusion module）模块融合高级语义特征与浅层特征。对网络进行性能评价，该网络测试mIoU为79.65%，FPS为59.6，在道路上进行实车实验，分割快速且效果良好。

关键词: 路沿分割, 双支路, 特征融合

SUN Yang, HAN Lei, WANG Chengqing, LI Yunpeng. Curb Segmentation Using Dual Branch and Feature Fusion Network[J]. Computer Engineering and Applications, 2023, 59(9): 255-261.

孙扬, 韩磊, 王程庆, 李韵鹏. 采用双支路与特征融合网络的路沿分割[J]. 计算机工程与应用, 2023, 59(9): 255-261.

References

[1] YANG W J，HUI Y T.Image scene analysis based on improved FCN model[J].International Journal of Pattern Recognition and Artificial Intelligence，2021，35（15）：2152020.
[2] 徐志京，汪毅.青光眼眼底图像的迁移学习分类方法[J].计算机工程与应用，2021，57（3）：144-149.
XU Z J，WANG Y.Glaucoma fundus images classification method based on transfer learning[J].Computer Engineering and Applications，2021，57（3）：144-149.
[3] LI J，JIANG F，YANG J，et al.Lane-DeepLab：lane semantic segmentation in automatic driving scenarios for high-definition maps[J].Neurocomputing，2021，465：15-25.
[4] ZHAO H，SHI J，QI X，et al.Pyramid scene parsing network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Jul 21-26，2017：2881-2890.
[5] GIRSHICK R，DONAHUE J，DARRELL T，et al.Region-based convolutional networks for accurate object detection and segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，38（1）：142-158.
[6] JIANG X，WANG F，CHEN Y，et al.RefineNet-based automatic delineation of the clinical target volume and organs at risk for three-dimensional brachytherapy for cervical cancer[J].Annals of Translational Medicine，2021，9（23）.
[7] AFIFY H M，MOHAMMED K，HASSANIEN A E.An improved framework for polyp image segmentation based on SegNet architecture[J].International Journal of Imaging Systems and Technology，2021，31（3）：1741-1751.
[8] RONNEBERGER O，FISCHER P，BROX T.U-Net：convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention，Munich，Oct 5?9，2015.Cham：Springer，2015：234-241.
[9] LI H，XIONG P，FAN H，et al.DFANet：deep feature aggregationfor real-time semantic segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：9522-9531.
[10] PASZKE A，CHAURASIA，KIM S，et al.ENet：a deepneural network architecture for real-time semantic segmentation[J].arXiv：1606.02147，2016.
[11] ORSIC M，KRESO I，BEVANDIC P，et al.In defense of pre-trained imagenet architectures for real-time semantic segmentation of road-driving images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：12607-12616.
[12] YU C，WANG J，PENG C，et al.Bisenet：bilateral segmentation network for real-time semantic segmentation[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：325-341.
[13] YU F，KOLTUN V.Multi-scale context aggregation by dilated convolutions[J].arXiv：1511.07122，2015.
[14] HU X，JING L，SEHAR U.Joint pyramid attention network for real-time semantic segmentation of urban scenes[J].Applied Intelligence，2021，52：580-594.
[15] YANG M，YU K，CHI Z，et al.Denseaspp for semantic segmentation in street scenes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：3684-3692.
[16] YU F，WANG D，SHELHAMER E，et al.Deep layer aggregation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：2403-2412.
[17] J′EGOU S，DROZDZAL M，V′AZQUEZ D，et al.The one hundred layers tiramisu：fully convolutional densenets for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2017：11-19.
[18] HU R，ROHRBACH M，DARRELL T.Segmentation from natural language expressions[C]//European Conference on Computer Vision.Cham：Springer，2016：108-124.
[19] 薛丽霞，钟欣，汪荣贵，等.基于深度特征融合的中低分辨率车型识别[J].计算机工程，2019，45（1）：233-238.
XUE L X，ZHONG X，WANG R G，et al.Mid-low resolution vehicle type recognition based on deep feature fusion[J].Computer Engineering，2019，45（1）：233-238.