Cross-Modality PET Synthesis Method Based on Residual and Adversarial Networks

doi:10.3778/j.issn.1002-8331.2008-0325

Abstract

Abstract: In order to solve the problem that the existing cross-modality image synthesis methods cannot capture the spatial and structural information of human tissues well, and the synthesized image has fuzzy edges and low signal-to-noise ratio, a cross-modality PET synthesis method based on residual module and generative adversarial networks is proposed. The algorithm introduces an improved residual inception module and attention mechanism into the generator network to enhance the feature learning ability and reduce the number of parameters. Multiscale discriminator is used to improve the discrimination performance. Multi-scale structural similarity loss is introduced into the loss function to better preserve the contrast information of the image. This algorithm is compared with several existing methods on the ADNI dataset. The experimental results show that the MAE of synthetic PET image is decreased, and the SSIM and PSNR are improved. Comparisons with the existing methods demonstrate that the improved model can retain the structure information of the image better and improve the quality of the synthetic image both visually and objectively.

Key words: cross-modality image synthesis, generative adversarial network, residual inception module, multi-scale discriminator

摘要： 针对现有跨模态图像合成方法不能很好地捕获人体组织的空间信息与结构信息，合成的图像具有边缘模糊、信噪比低等问题，提出一种融合残差模块和生成对抗网络的跨模态PET图像合成方法。该算法在生成器网络中引入改进的残差初始模块和注意力机制，减少参数量的同时增强了生成器的特征学习能力。判别器采用多尺度判别器，以提升判别性能。损失函数中引入多层级结构相似损失，以更好地保留图像的对比度信息。该算法在ADNI数据集上与主流算法进行对比，实验结果表明，合成PET图像的MAE指标有所下降，SSIM与PSNR指标有所提升。实验结果显示，提出的模型能很好地保留图像的结构信息，在视觉和客观指标上都能提高合成图像的质量。

关键词: 跨模态图像合成, 生成对抗网络, 残差初始模块, 多尺度判别器

XIAO Chenchen, CHEN Legeng, WANG Shuqiang. Cross-Modality PET Synthesis Method Based on Residual and Adversarial Networks[J]. Computer Engineering and Applications, 2022, 58(1): 218-223.

肖晨晨, 陈乐庚, 王书强. 融合残差和对抗网络的跨模态PET图像合成方法[J]. 计算机工程与应用, 2022, 58(1): 218-223.

References

[1] KANG Y.Medical imaging technology and system[M].Beijing：Tsinghua University Press，2014.
[2] LI R，ZHANG W，SUK H I，et al.Deep learning based imaging data completion for improved brain disease diagnosis[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention，2014：305-312.
[3] GAO F，WU T，CHU X，et al.Deep residual inception encoder-decoder network for medical imaging synthesis[J].IEEE Journal of Biomedical and Health Informatics，2019，24（1）：39-49.
[4] NIE D，TRULLO R，LIAN J，et al.Medical image synthesis with context-aware generative adversarial networks[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention，2017：417-425.
[5] BI L，KIM J，KUMAR A，et al.Synthesis of positron emission tomography（PET） images via multi-channel generative adversarial networks（GANs）[M]//Molecular imaging，recon-struction and analysis of moving body organs，and stroke imaging and treatment.Cham：Springer，2017：43-51.
[6] BEN-COHEN A，KLANG E，RASKIN S P，et al.Cross-modality synthesis from CT to PET using FCN and GAN networks for improved automated lesion detection[J].Engineering Applications of Artificial Intelligence，2019，78：186-194.
[7] YAAKUB S N，MCGINNITY C J，CLOUGH J R，et al.Pseudo-normal PET synthesis with generative adversarial networks for localising hypometabolism in epilepsies[C]//International Workshop on Simulation and Synthesis in Medical Imaging，2019：42-51.
[8] GOODFELLOW I，POUGET-ABADIE J，MIRZA M，et al.Generative adversarial nets[C]//Advances in Neural Information Processing Systems，2014：2672-2680.
[9] MIRZA M，OSINDERO S.Conditional generative adversarial nets[J].arXiv：1411.1784，2014.
[10] ISOLA P，ZHU J Y，ZHOU T，et al.Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1125-1134.
[11] OKTAY O，SCHLEMPER J，FOLGOC L L，et al.Attention U-net：Learning where to look for the pancreas[J].arXiv：1804.03999，2018.
[12] RONNEBERGER O，FISCHER P，BROX T.U-net：Convolutional networks for biomedical image segmentation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention，2015：234-241.
[13] WANG Z，SIMONCELLI E P，BOVIK A C.Multiscale structural similarity for image quality assessment[C]//Proceedings of the Thrity-Seventh Asilomar Conference on Signals，Systems & Computers，2003：1398-1402.
[14] WEINER M W，VEITCH D P，AISEN P S，et al.Impact of the Alzheimer’s disease neuroimaging initiative，2004 to 2014[J].Alzheimer’s Dement，2015，11（7）：865-884.
[15] WOOLRICH M W，JBADBI S，PATENAUDE B，et al.Bayesian analysis of neuroimaging data in FSL[J].Neu-roImage，2009，45：173-186.
[16] DAR S U H，YURT M，KARACAN L，et al.Image synthesis in multi-contrast MRI with conditional generative adversarial networks[J].IEEE Transactions on Medical Imaging，2019，38（10）：2375-2388.
[17] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.