Review of Application of Transfer Learning in Medical Image Field

doi:10.3778/j.issn.1002-8331.2107-0300

Abstract

Abstract:

Deep learning technology has developed rapidly and achieved significant results in the field of medical image treatment. However, due to the small number of medical image samples and difficult annotation, the effect of deep learning is far from reaching the expectation. In recent years, using transfer learning method to alleviate the problem of insufficient medical image samples and improve the effect of deep learning technology in the field of medical image has become one of the research hotspots. This paper first introduces the basic concepts, types, common strategies and models of transfer learning methods, then combs and summarizes the representative related research in the field of medical images according to the types of transfer learning methods, and finally summarizes and prospects the future development of this field.

Key words: medical image, transfer learning, neural network, deep learning

摘要：

深度学习技术发展迅速，在医学图像处理领域取得了显著成果。但是由于医学图像样本少，标注困难，使得深度学习的效果远未达到预期。近年，利用迁移学习方法缓解医学图像样本不足的问题，提高深度学习技术在医学图像领域的效果，成为了研究热点之一。介绍了迁移学习方法的基本概念、类型、常用策略及模型，根据迁移学习方法的类型，对当前医学图像领域具有代表性的相关研究进行了梳理与小结，对该领域的未来发展进行了总结和展望。

关键词: 医学图像, 迁移学习, 神经网络, 深度学习

GAO Shuang, XU Qiaozhi. Review of Application of Transfer Learning in Medical Image Field[J]. Computer Engineering and Applications, 2021, 57(24): 39-50.

高爽，徐巧枝. 迁移学习方法在医学图像领域的应用综述[J]. 计算机工程与应用, 2021, 57(24): 39-50.

References

[1] OQUAB M，BOTTOU L，LAPTEV I，et al.Learning and transferring mid-level image representations using convolutional neural networks[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition，2014：1717-1724.
[2] WEISS K，KHOSHGOFTAAR T M，WANG D D.A survey of transfer learning[J].Journal of Big Data，2016，3（1）：1-40.
[3] YI Y，DORETTO G.Boosting for transfer learning with multiple sources[C]//The Twenty-Third IEEE Conference on Computer Vision and Pattern Recognition，2010：13-18.
[4] OZBULAK G，AYTAR Y，EKENEL H K.How transferable are CNN-based features for age and gender classification?[C]//2016 International Conference of the Biometrics Special Interest Group（BIOSIG），2016：1-6.
[5] MUSTAFA B，LOH A，FREYBERG J，et al.Supervised transfer learning at scale for medical imaging[J].arXiv：2101.05913，2021.
[6] LECUN Y，BENGIO Y，HINTON G.Deep learning[J].Nature，2015，521（7553）：436-444.
[7] SHIN H C，ROTH H R，GAO M，et al.Deep convolutional neural networks for computer-aided detection：CNN architectures，dataset characteristics and transfer learning[J].IEEE Transactions on Medical Imaging，2016，35（5）：1285-1298.
[8] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.Imagenet classification with deep convolutional neural networks[J].Advances in Neural Information Processing Systems，2012，25：1097-1105.
[9] SIMONYAN K，ZISSERMAN A.Very deep convolutional networks for large scale image recognition[J].arXiv：1409.
1556，2014.
[10] SZEGEDY C，LIU W，JIA Y，et al.Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：1-9.
[11] HE K，ZHANG X，REN S，et al.Identity mappings in deep residual networks[C]//European Conference on Computer Vision.Cham：Springer，2016：630-645.
[12] HUANG G，LIU Z，VAN DER MAATEN L，et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：4700-4708.
[13] HOWARD A，SANDLER M，CHU G，et al.Searching for mobile Netv3[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：1314-1324.
[14] ABBAS A，ABDELSAMEA M M，GABER M M.Detrac：transfer learning of class decomposed medical images in convolutional neural networks[J].IEEE Access，2020，8：74901-74913.
[15] MAGHDID H S，ASAAD A T，GHAFOOR K Z，et al.Diagnosing COVID-19 pneumonia from X-ray and CT images using deep learning and transfer learning algorithms[C]//Multimodal Image Exploitation and Learning 2021 International Society for Optics and Photonics，2021：117340.
[16] GAO F，YOON H，WU T，et al.A feature transfer enabled multi-task deep learning model on medical imaging[J].Expert Systems with Applications，2020，143：112957.
[17] DAWUD A M，YURTKAN K，OZTOPRAK H.Application of deep learning in neuroradiology：brain haemorrhage classification using transfer learning[J].Computational Intelligence and Neuroscience，2019（6）：1-12.
[18] YANG Y，LI X，WANG P，et al.Multi-source transfer learning via ensemble approach for initial diagnosis of alzheimer’s disease[J].IEEE Journal of Translational Engineering in Health and Medicine，2020，8：1-10.
[19] WAHAB N，KHAN A，LEE Y S.Transfer learning based deep CNN for segmentation and detection of mitoses in breast cancer histopathological images[J].Microscopy，2019，68（3）：216-233.
[20] LIU W，CHENG Y，LIU Z，et al.Preoperative prediction of Ki-67 status in breast cancer with multiparametric MRI using transfer learning[J].Academic Radiology，2021，28（2）：44-53.
[21] PATRINI I，RUPERTI M，MOCCIA S，et al.Transfer learning for informative-frame selection in laryngoscopic videos through learned features[J].Medical & Biological Engineering Computing，2020，58（6）：1225-1238.
[22] KHAN S U，ISLAM N，JAN Z，et al.A novel deep learning based framework for the detection and classification of breast cancer using transfer learning[J].Pattern Recognition Letters，2019，125（7）：1-6.
[23] HUANG X，LEI Q，XIE T，et al.Deep transfer convolutional neural network and extreme learning machine for lung nodule diagnosis on CT images[J].Knowledge-Based Systems，2020，204：106230.
[24] 黄鸿，彭超，吴若愚，等.基于部分注释CT图像的自监督迁移学习肺结节分类[J].光学学报，2020，40（18）：93-100.
HUANG H，PENG C，WU R Y，et al.Self-supervised transfer learning of pulmonary nodule classification based on partially annotated CT images[J].Acta Optica Sinica，2020，40（18）：93-100.
[25] MISRA S，JEON S，LEE S.Multi-channel transfer learning of chest x-ray images for screening of COVID-19[J].Electronics，2020，9（9）：1388.
[26] JIN B，CRUZ L，GONCALVES N.Deep facial diagnosis：deep transfer learning from face recognition to facial diagnosis[J].IEEE Access，2020，8：123649-123661.
[27] CHEN S，MA K，ZHENG Y.Med3d：transfer learning for 3d medical image analysis[J].arXiv：1904.00625，2019.
[28] ZHOU Z，SODHA V，SIDDIQUEE M M R，et al.Models genesis：generic autodidactic models for 3d medical image analysis[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2019：384-393.
[29] HUANG H，CHEN H，XU H，et al.Cross-tissue/organ transfer learning for the segmentation of ultrasound images using deep residual U-Net[J].Journal of Medical and Biological Engineering，2021，41（2）：137-145.
[30] GUAN H，LIU M.Domain adaptation for medical image analysis：a survey[J].arXiv：2102.09508，2021.
[31] INDRASWARI R，KURITA T，ARIFIN A Z，et al.Multi-projection deep learning network for segmentation of 3D medical images[J].Pattern Recognition Letters，2019，125（7）：791-797.
[32] DU R，VARDHANABHUTI V.3D-RADNet：extracting labels from DICOM metadata for training general medical domain deep 3D convolution neural networks[C]//Proceedings of International Conference on Medical Imaging with Deep Learning，2020：174-192.
[33] ZHOU H Y，YU S，BIAN C，et al.Comparing to learn：surpassing imagenet pretraining on radiographs by comparing image representations[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2020：398-407.
[34] LI W，ZHAO Y，CHEN X，et al.Detecting Alzheimer’s disease on small dataset：a knowledge transfer perspective[J].IEEE Journal of Biomedical and Health Informatics，2018，23（3）：1234-1242.
[35] VIRK J S，BATHULA D R.Domain-specific，semi-supervised transfer learning for medical imaging[C]//CODS COMAD 2021：8th ACM IKDD CODS and 26th COMAD，2021.
[36] MANZO M，PELLINO S.Bucket of deep transfer learning features and classification models for melanoma detection[J].Journal of Imaging，2020，6（12）：129.
[37] LIAO X，QIAN Y，CHEN Y，et al.MMTLNet：multi-modality transfer learning network with adversarial training for 3D whole heart segmentation[J].Computerized Medical Imaging and Graphics，2020，85：101785.
[38] 肖儿良，周莹，简献忠.迁移学习与GAN结合的医学图像融合模型[J].小型微型计算机系统，2020，41（9）：1972-1978.
XIAO E L，ZHOU Y，JIAN X Z.Medical image fusion model combining transfer learning and generative adversarial network[J].Journal of Chinese Computer Systems，2020，41（9）：1972-1978.
[39] GULRAJANI I，AHMED F，ARJOVSKY M，et al.Improved training of wasserstein gans[J].arXiv：1704.00028，2017.
[40] 黄玄曦，张健毅，杨涛，等.生成对抗网络在医学图像生成中的应用[J].北京电子科技学院学报，2020，28（4）：36-48.
HUANG X X，ZHANG J Y，YANG T，et al.Application of generative adversarial network in medical image generation[J].Journal of Beijing Electronic Science and Technology Institute，2020，28（4）：36-48.
[41] CHA S M，LEE S S，KO B.Attention-based transfer learning for efficient pneumonia detection in chest X-ray images[J].Applied Sciences，2021，11（3）：1242.
[42] TAN C，SUN F，KONG T，et al.Attention-based transfer learn ing for brain-computer interface[C]//ICASSP 2019-2019 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP），2019：1154-1158.
[43] GUY-FERNAND K N，ZHAO J J，SABUNI F M，et al.Classification of brain tumor leveraging goal-driven visual attention with the support of transfer learning[C]//2020 Information Communication Technologies Conference（ICTC），2020：328-332.
[44] WANG Q，WU B，ZHU P，et al.ECA-Net：efficient channel attention for deep convolutional neural networks[C]//2020 IEEE CVF Conference on Computer Vision and Pattern Recognition（CVPR），2020.
[45] ZHANG Q L，YANG Y B.Sa-net：shuffle attention for deep convolutional neural networks[C]//ICASSP 2021-2021 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP），2021：2235-2239.
[46] ZHUANG X，LI Y，HU Y，et al.Self-supervised feature learning for 3D medical images by playing a Rubik’s cube[C]//International Conference on Medical Image Computing and Computer Assisted Intervention，2019：420-428.
[47] ZHU J，LI Y，HU Y，et al.Rubik’s Cube+：a self-supervised feature learning framework for 3D medical image analysis[J].Medical Image Analysis，2020，64：101746.
[48] TAO X，LI Y，ZHOU W，et al.Revisiting Rubik’s cube：self-supervised learning with volume-wise transformation for 3D medical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2020：238-248.
[49] LI X，YU L，CHEN H，et al.Transformation-consistent self-ensembling model for semisupervised medical image segmentation[J].IEEE Transactions on Neural Networks and Learning Systems，2020，32（2）：523-534.
[50] 毛莺池，唐江红，王静，等.基于Faster R-CNN的多任务增强裂缝图像检测方法[J].智能系统学报，2021，16（2）：286-293.
MAO Y C，TANG J H，WANG J，et al.Multi-task enhanced dam crack image detection based on Faster R-CNN[J].CAAI Transactions on Intelligent Systems，2021，16（2）：286-293.
[51] HEKER M，GREENSPAN H.Joint liver lesion segmentation and classification via transfer learning[J].arXiv：2004.12352，2020.