Application of Improved Mask R-CNN Network in Medical Image Recognition and Segmentation

doi:10.3778/j.issn.1002-8331.2105-0092

Abstract

Abstract:

Aiming at the deficiencies of the existing medical image processing methods in the segmentation of human body complex structures, tissues and organs. A Mask R-CNN network that reuses low-level feature information is proposed, which can segment specific organs at the same time. In order to improve the utilization of the lower-level feature layer that contains more detailed information, the lower-level feature information is added to the high-level features, so that the low-level and high-level features complement each other. The characteristic layer of the original image after the first length and width compression twice is defined as the C1 layer, and then it is implemented by two methods of multiplexing the C1 layer and multiplexing the successively convolved C1 layer. And the backbone network is streamlined to speed up the network training speed and reduce the recognition and segmentation time. The mandible is used as the application object. A self-built data set containing 1 064 mandibular CT images is divided into a training set and a validation set at a ratio of 9∶1 for training, and then the Mask R-CNN network that convolves the C1 layer in sequence is reused. The training loss is reduced to 2.8%, and the verification loss is reduced to 6.6%, it indicates that the network has a high accuracy in the recognition and segmentation of the mandible.

Key words: neural network, feature fusion, medical image processing, mandible recognition and segmentation, time costs

摘要：

针对现有医学图像处理方法在人体复杂结构组织器官分割中的不足，提出复用低层特征信息的Mask R-CNN网络。该网络可对特定组织器官识别时同时进行分割，为了提高包含较多细节信息的低层特征层的利用率，将低层的特征信息添加到高层的特征中，使低层与高层特性优劣互补，将原始图像首次长宽压缩两次后的特征层定义为C1层，而后分别通过复用C1层和复用依次卷积的C1层这两种方法实现。并将主干网络进行了精简，以加快网络的训练速度，降低识别和分割的时间。以下颌骨作为应用对象，自建包含1?064张下颌骨CT图片的数据集，按9∶1的比例划分为训练集和验证集进行训练，使得复用依次卷积C1层的Mask R-CNN网络的训练损失降至2.8%，验证损失降至6.6%，表明该网络在下颌骨的识别和分割上具有很高的准确率。

关键词: 神经网络, 特征融合, 医学图像处理, 下颌骨识别与分割, 时间成本

LU Wei, LIU Dan, SHAO Min, WU Yangdong. Application of Improved Mask R-CNN Network in Medical Image Recognition and Segmentation[J]. Computer Engineering and Applications, 2021, 57(24): 234-241.

卢苇，刘丹，邵敏，吴扬东. 改进Mask R-CNN网络在医学图像识别与分割中的应用[J]. 计算机工程与应用, 2021, 57(24): 234-241.

References

[1] 孙文燕，董恩清，曹祝楼，等.一种基于模糊主动轮廓的鲁棒局部分割方法[J].自动化学报，2017，43（4）：611-621.
SUN W Y，DONG E Q，CAO Z L，et al.A robust local segmentation method based on fuzzy-energy based active contour[J].Acta Automatica Sinica，2017，43（4）：611-621.
[2] VAN EIJNATTEN M，VAN DIJK R，DOBBE J，et al.CT image?segmentation?methods?for?bone?used?in medical?additive?manufacturing[J].Medical Engineering & Physics，2018，51：6-16.
[3] FILIPPOU V，TSOUMPAS C.Recent advances on the development of phantoms using 3D printing for imaging with CT，MRI，PET，SPECT，and ultrasound[J].Medical Physics，2018，45（9）：740-760.
[4] SHEN Y，MA C Y，WANG L，et al.Surgical management of giant cell tumors in temporomandibular joint region involving lateral skull base：a multidisciplinary approach[J].Journal of Oral and Maxillofacial Surgery，2016，74（11）：2295-2311.
[5] 田萱，王亮，丁琪.基于深度学习的图像语义分割方法综述[J].软件学报，2019，30（2）：250-278.
TIAN X，WANG L，DING Q.Review of image semantic segmentation based on deep learning[J].Journal of Software，2019，30（2）：250-278.
[6] SHELHAMER E，LONG J，DARRELL T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（4）：640-651.
[7] 钱宝鑫，肖志勇，宋威.改进的卷积神经网络在肺部图像上的分割应用[J].计算机科学与探索，2020，14（8）：1358-1367.
QIAN B X，XIAO Z Y，SONG W.Application of improved convolutional neural network in lung image segmentation[J].Journal of Frontiers of Computer Science and Technology，2020，14（8）：1358-1367.
[8] 马其鹏，谢林柏，彭力.改进的卷积神经网络在医学影像分割中的应用[J].激光与光电子学进展，2020，57（14）：190-196.
MA Q P，XIE L B，PENG L.Application of improved convolutional neural network in medical image segmentation[J].Laser & Optoel Optoelectronics Progress，2020，57（14）：190-196.
[9] 田娟秀，刘国才，谷珊珊，等.医学图像分析深度学习方法研究与挑战[J].自动化学报，2018，44（3）：401-424.
TIAN J X，LIU G C，GU S S，et al.Deep learning in medical image analysis and its challenges[J].Acta Automatica Sinica，2018，44（3）：401-424.
[10] LIN T Y，DOLLáR P，GIRSHICK R，et al.Feature pyramid networks for object detection[C]//Conference on Computer Vision and Pattern Recognition，2016：936-944.
[11] REN S，HE K M，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis ＆ Machine Intelligence，2015，39（6）：1137-1149.
[12] HE K M，GKIOXARI G，DOLLáR P，et al.Mask R-CNN[C]//2017 IEEE International Conference on Computer Vision（ICCV），2017.
[13] 于宁波，刘嘉男，高丽，等.基于深度学习的膝关节MR图像自动分割方法[J].仪器仪表学报，2020，41（6）：140-149.
YU N B，LIU J N，GAO L，et al.Auto-segmentation method based on deep learning for the knee joint in MR images[J].Chinese Journal of Scientific Instrument，2020，41（6）：140-149.
[14] HE K M，ZHANG X，REN S.et al.Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2016：770-778.
[15] 黄毅鹏，胡冀苏，钱旭升，等.SE-Mask-RCNN：多参数MRI前列腺癌分割方法[J].浙江大学学报（工学版），2021，55（1）：203-212.
HUANG Y P，HU J S，QIAN X S，et al.SE-Mask-RCNN：segmentation method for prostate cancer on multi-parametric MRI[J].Journal of Zhejiang University（Engineering Science），2021，55（1）：203-212.
[16] 石杰，周亚丽，张奇志.基于改进Mask RCNN和Kinect的服务机器人物品识别系统[J].仪器仪表学报，2019，40（4）：216-228.
SHI J，ZHOU Y L，ZHANG Q Z.Service robot item recognition system based on improved Mask RCNN and Kinect[J].Chinese Journal of Scientific Instrument，2019，40（4）：216-228.
[17] ROSS G.Fast R-CNN[C]//IEEE International Conference on Computer Vision（ICCV），2015：1440-1448.