Improved Mask R-CNN Method for Thyroid Nodules Segmentation in Ultrasound Images

doi:10.3778/j.issn.1002-8331.2101-0032

Abstract

Abstract: The ultrasound image of thyroid nodules has low contrast and severe speckle noise. The morphology of thyroid nodules varies greatly from different patients, which makes it extremely difficult for doctors to accurately segment the nodules. In order to accurately segment thyroid nodules from ultrasound images, this paper improves the backbone network of Mask R-CNN（mask region-convolutional neural network）. The attention mechanism module is added to the residual network layer of the original backbone network to improve the convergence of the model, and a branch from the bottom to the top is added to the feature pyramid network. After the branch output feature map is merged, it is input to the region proposal network and the region of interesting align to integrate multi-scale features while balancing the difference in feature map information. After testing 600 ultrasound images of thyroid nodules, the improved Mask R-CNN has an average Dice coefficient of 0.914 8, an average accuracy of 0.932 2, an average recall rate of 0.903 4, and an average F1 score of 0.917 6. The Dice coefficient is 0.080 6 higher than the original Mask R-CNN. The improved algorithm can be applied to automatically segment ultrasound images of thyroid nodules in actual clinical medicine.

Key words: thyroid nodule ultrasound image, Mask R-CNN, backbone network, image segmentation

摘要： 甲状腺结节超声图像对比度低，斑点噪声严重，且不同病人的甲状腺结节形态差异较大，这给医生准确分割结节带来极大困难。为了精确地从超声图像中分割出甲状腺结节，对原掩膜区域卷积神经网络（mask region-convolutional neural network，Mask R-CNN）的主干网络进行改进。在原主干网络的残差网络层中加入注意力机制模块来提高模型收敛性，并且在特征金字塔网络中增添一条由下向上的支路，将该支路输出特征图进行融合后，输入至区域推荐网络和感兴趣区域池化层，从而能够在融合多尺度特征的同时平衡特征图信息差异。经过对600幅甲状腺结节超声图像进行测试，改进后Mask R-CNN图像分割的平均Dice系数为0.914?8，平均精确度为0.932?2，平均召回率为0.903?4，平均F1分数为0.917?6。改进算法分割的Dice系数比原Mask R-CNN提升了0.080?6，改进算法可以应用于实际临床医学中自动分割甲状腺结节超声图像。

关键词: 甲状腺结节超声图像, Mask R-CNN, 主干网络, 图像分割

LIU Mingkun, ZHANG Junhua, LI Zonggui. Improved Mask R-CNN Method for Thyroid Nodules Segmentation in Ultrasound Images[J]. Computer Engineering and Applications, 2022, 58(16): 219-225.

刘明坤, 张俊华, 李宗桂. 改进Mask R-CNN的甲状腺结节超声图像分割方法[J]. 计算机工程与应用, 2022, 58(16): 219-225.

References

[1] HERMUS A R，HUYSMANS D A.Treatment of benign nodular thyroid disease[J].The New England Journal of Medicine，1998，338（20）：1438-1447.
[2] MCGUIRE S.World cancer report 2014.Geneva，Switzerland：world health organization，international agency for research on cancer，WHO press，2015[J].Advances in Nutrition，2016，7（2）：418-419.
[3] PACINI F，SCHLUMBERGER M，DRALLE H，et al.European consensus for the management of patients with differentiated thyroidcarcinoma of the follicular epithelium[J].European Journal of Endocrinology，2006，154（6）：787-803.
[4] 董芬，张彪，单广良.中国甲状腺癌的流行现状和影响因素[J].中国癌症杂志，2016，26（1）：47-52.
DONG F，ZHANG B，SHAN G L.Distribution and risk factors of thyroid cancer in China[J].China Oncology，2016，16（1）：47-52.
[5] 邹奕轩，周蕾蕾，赵紫婷，等.基于卷积神经网络的甲状腺结节超声图像良恶性分类研究[J].中国医学装备，2020，17（3）：9-13.
ZHOU Y X，ZHOU L L，ZHAO Z T，et al.Study on the classification of benign and malignant thyroid nodule in ultrasound image on the basis of CNNs[J].Chinese Medical Equipment，2020，17（3）：9-13.
[6] 刘宇，陈胜.医学图像分割方法综述[J].电子科技，2017，30（8）：169-172.
LIU Y，CHEN S.Review of medical image segmentation method[J].Electronic Science and Technology，2017，30（8）：169-172.
[7] KOUNDAL D，GUPTA S，SINGH S.Computer aided thyroid nodule detection system using medical ultrasound images[J].Biomedical Signal Processing and Control，2018，40：117-130.
[8] MA J，WU F，JIANG T，et al.Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks[J].International Journal of Computer Assisted Radiology and Surgery，2017，12（11）：1895-1910.
[9] LI X，WANG S，WEI X，et al.Fully convolutional networks for ultrasound image segmentation of thyroid nodules[C]//2018 IEEE 20th International Conference on High Performance Computing and Communications，IEEE 16th International Conference on Smart City，IEEE 4th International Conference on Data Science and Systems，2018：886-890.
[10] WANG Y，WEI K，WAN P.A method of ultrasonic image recognition for thyroid papillary carcinoma based on deep convolution neural network[J].NeuroQuantology，2018，16（5）：757-768.
[11] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems，2015：91-99.
[12] YING X，YU Z，YU R，et al.Thyroid nodule segmentation in ultrasound images based on cascaded convolutional neural network[C]//25th International Conference on Neural Information Processing.Cham：Springer，2018：373-384.
[13] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[14] LIN T Y，DOLLáR P，GIRSHICK R，et al.Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：2117-2125.
[15] HU J，SHEN L，SUN G.Squeeze-and-excitation networks[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：7132-7141.
[16] WOO S，PARK J，LEE J Y，et al.CBAM：convolutional block attention module[C]//15th European Conference on Computer Vision，2018：3-19.
[17] LUC P，COUPRIE C，LECUN Y，et al.Predicting future instance segmentation by forecasting convolutional features[C]//15th European Conference on Computer Vision，2018.
[18] SHATTUCK D W，SANDOR-LEAHY S R，SCHAPER K A，et al.Magnetic resonance image tissue classification using a partial volume model[J].NeuroImage，2001，13（5）：856-876.