医学图像分割中的双分支特征提取器及高效特征融合方法

doi:10.3778/j.issn.1002-8331.2404-0217

摘要/Abstract

摘要： 在医学图像分割领域，卷积网络和Transformer网络均以其独特优势而备受青睐，但各自的应用也面临着特定的局限性。此外，现有的特征融合模块存在显著的信息损失，无法充分学习和利用空间和通道之间的复杂关系来实现更准确的分割。为此，提出了一种双分支并行网络特征提取器，解决了单个网络在信息提取方面的不足，有效地克服了两个网络串联组合时可能出现的信息瓶颈问题。同时，为了更充分地利用空间和通道之间的复杂关系，进一步引入了多分支局部全局特征融合增强模块，它能够高效地融合双分支的特征。实验表明，该算法在Synapse和ACDC数据集上表现出色，平均Dice分别达到83.32%和91.82%，HD95指标分别达到15.80 mm和1.29 mm，具有较强的竞争力。

关键词: 医学图像分割, 卷积神经网络（CNN）, Transformer网络, 特征融合

Abstract: In medical image segmentation, convolutional and Transformer networks are highly favored for their unique advantages, but their respective applications also face specific limitations. In addition, existing feature fusion modules suffer from significant information loss and cannot fully learn and utilize the complex relationships between space and channels to achieve more accurate segmentation. To this end, a dual branch parallel network feature extractor is first proposed, which solves the shortcomings of a single network in information extraction and effectively overcomes the information bottleneck problem that may occur when two networks are combined in series. Meanwhile, to fully utilize the complex relationship between space and channels, this paper further introduces a multi-branch local global feature fusion enhancement module, which can efficiently fuse features from both branches. The experiments show that the algorithm performs well on the Synapse and ACDC datasets, with an average Dice of 83.32% and 91.82%, and an HD95 index of 15.80 mm and 1.29 mm, respectively, demonstrating strong competitiveness.

Key words: medical image segmentation, convolutional neural network (CNN), Transformer networks, feature fusion

张凡, 侯惠芳, 张自豪, 潘泉. 医学图像分割中的双分支特征提取器及高效特征融合方法[J]. 计算机工程与应用, 2025, 61(14): 286-296.

ZHANG Fan, HOU Huifang, ZHANG Zihao, PAN Quan. Dual Branch Feature Extractor and Efficient Feature Fusion Method in Medical Image Segmentation[J]. Computer Engineering and Applications, 2025, 61(14): 286-296.

参考文献

[1] HESAMIAN M H, JIA W J, HE X J, et al. Deep learning techniques for medical image segmentation: achievements and challenges[J]. Journal of Digital Imaging, 2019, 32(4): 582-596.
[2] FU Y B, LEI Y, WANG T H, et al. A review of deep learning based methods for medical image multi-organ segmentation[J]. Physica Medica, 2021, 85: 107-122.
[3] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2015: 234-241.
[4] 崔珂, 田启川, 廉露. 基于U-Net变体的医学图像分割算法综述[J]. 计算机工程与应用, 2024, 60(11): 32-49.
CUI K, TIAN Q C, LIAN L. Review of medical image segmentation algorithms based on U-Net variants[J]. Computer Engineering and Applications, 2024, 60(11): 32-49.
[5] ZHOU Z, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet++: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(6): 1856-1867.
[6] DIAKOGIANNIS F I, WALDNER F, CACCETTA P, et al. ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 162: 94-114.
[7] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000-6010.
[8] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[J]. arXiv:2010.11929, 2020.
[9] CAO H, WANG Y, CHEN J, et al. Swin-UNet: UNet-like pure transformer for medical image segmentation[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 205-218.
[10] WANG W X, CHEN C, DING M, et al. TransBTS: multimodal brain tumor segmentation using transformer[C]//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2021: 109-119.
[11] HUANG X H, DENG Z F, LI D D, et al. MissFormer: an effective medical image segmentation transformer[J]. arXiv: 2109. 07162, 2021.
[12] CHEN J N, LU Y Y, YU Q H, et al. TransUNet: transformers make strong encoders for medical image segmentation[J]. arXiv:2102.04306, 2021.
[13] CHEN B Z, LIU Y S, ZHANG Z, et al. TransAttUnet: multi-level attention-guided U-Net with transformer for medical image segmentation[J]. IEEE Transactions on Emerging Topics in Computational Intelligence, 2024, 8(1): 55-68.
[14] HEIDARI M, KAZEROUNI A, SOLTANY M, et al. HiFormer: hierarchical multi-scale representations using transformers for medical image segmentation[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2023: 6191-6201.
[15] AZAD R, JIA Y W, AGHDAM E K, et al. Enhancing medical image segmentation with TransCeption: a multi-scale feature fusion approach[J]. arXiv:2301.10847, 2023.
[16] FENG S L, ZHAO H M, SHI F, et al. CPFNet: context pyramid fusion network for medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(10): 3008-3018.
[17] XU Y, ZHOU Z, LI X, et al. FFU-Net: feature fusion U-Net for lesion segmentation of diabetic retinopathy[J]. BioMed Research International, 2021, 2021: 6644071.
[18] SUN Y H, DAI D W, ZHANG Q N, et al. MSCA-Net: multi-scale contextual attention network for skin lesion segmentation[J]. Pattern Recognition, 2023, 139: 109524.
[19] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 3431-3440.
[20] ZHOU Z, SIDDIQUEE R M M, TAJBAKHSH N, et al. UNet++: a nested U-Net architecture for medical image segmentation[C]//Proceedings of the Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, 2018: 3-11.
[21] HUANG H M, LIN L F, TONG R F, et al. UNet 3+: a full-scale connected UNet for medical image segmentation[C]//Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2020: 1055-1059.
[22] OKTAY O, SCHLEMPER J, LE FOLGOC L, et al. Attention U-Net: learning where to look for the pancreas[J]. arXiv:1804.03999, 2018.
[23] SINHA A, DOLZ J. Multi-scale self-guided attention for medical image segmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(1): 121-130.
[24] LIN A L, CHEN B Z, XU J Y, et al. DS-TransUNet: dual swin transformer U-Net for medical image segmentation[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-15.
[25] HATAMIZADEH A, TANG Y C, NATH V, et al. UNETR: transformers for 3D medical image segmentation[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2022: 1748-1758.
[26]GAO Y, ZHOU M, METAXAS D N. UTNet: a hybrid transformer architecture for medical image segmentation[C]//Proceedings of the 24th International Conference on Medical Image Computing and Computer Assisted Intervention. Cham: Springer International Publishing, 2021: 61-71.
[27] ZHOU H Y, GUO J S, ZHANG Y H, et al. nnFormer: interleaved transformer for volumetric segmentation[J]. arXiv:2109.03201, 2021.
[28] XIE Y, ZHANG J, SHEN C, et al. CoTr: efficiently bridging CNN and transformer for 3D medical image segmentation[C]//Proceedings of the 24th International Conference on Medical Image Computing and Computer Assisted Intervention, 2021: 171-180.
[29] HUO X Z, SUN G, TIAN S W, et al. HiFuse: hierarchical multi-scale feature fusion network for medical image classification[J]. Biomedical Signal Processing and Control, 2024, 87: 105534.
[30] ZHANG B, WANG Y, DING C F, et al. Multi-scale feature pyramid fusion network for medical image segmentation[J]. International Journal of Computer Assisted Radiology and Surgery, 2023, 18(2): 353-365.
[31] SRIVASTAVA A, JHA D, CHANDA S, et al. MSRF-Net: a multi-scale residual fusion network for biomedical image segmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26(5): 2252-2263.
[32] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
[33] SHEN Z R, ZHANG M Y, ZHAO H Y, et al. Efficient attention: attention with linear complexities[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2021: 3530-3538.
[34] ALI A, TOUVRON H, CARON M, et al. XCiT: cross-covariance image transformers[J]. Advances in Neural Information Processing Systems, 2021, 34: 20014-20027.
[35] WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 11531-11539.
[36] SCHLEMPER J, OKTAY O, SCHAAP M, et al. Attention gated networks: learning to leverage salient regions in medical images[J]. Medical Image Analysis, 2019, 53: 197-207.
[37] SUN J D, ZHAO J Q, WU X S, et al. DSGA-Net: deeply separable gated transformer and attention strategy for medical image segmentation network[J]. Journal of King Saud University-Computer and Information Sciences, 2023, 35(5): 101553.
[38] LI J, CHEN N, ZHOU H, et al. MCRformer: morphological constraint reticular transformer for 3D medical image segmentation[J]. Expert Systems with Applications, 2023, 232: 120877.
[39] HUANG T Y, CHEN J X, JIANG L F. DS-UNeXt: depthwise separable convolution network with large convolutional kernel for medical image segmentation[J]. Signal, Image and Video Processing, 2023, 17(5): 1775-1783.
[40] WANG J, ZHAO H Y, LIANG W, et al. Cross-convolutional transformer for automated multi-organs segmentation in a variety of medical images[J]. Physics in Medicine & Biology, 2023, 68(3): 035008.