基于卷积神经网络的扁桃体咽拭子采样机器人

doi:10.3778/j.issn.1002-8331.2110-0386

摘要/Abstract

摘要： 针对医务人员咽拭子采样时有较高被传染风险，提出咽拭子采样机器人。采样需检测、识别和定位扁桃体的位置信息，由于扁桃体在原图中尺寸较小、特征提取困难，设计基于卷积神经网络的两步检测模型。第一步，使用口腔检测模块将口腔图像从自制的人体面部数据集中检测并切割出来；第二步，在口腔图像基础上，识别位扁桃体的位置信息。结合深度图像计算扁桃体在机器人世界坐标系的坐标，控制机械臂运动到指定位置，实现咽拭子采样。实验证明，系统使用基于卷积神经网络的扁桃体两步检测模型具有较高的准确性和检测效率，检测结果的AP50和检测平均时间均优于对比算法且能够准确地完成咽拭子采样。

关键词: 目标识别, 检测定位, 深度学习, 医用辅助机器人

Abstract: In view of the high risk of infection in throat swab sampling of medical staff, this paper proposes a throat swab sampling robot. Sampling needs to detect, identify and locate the position information of tonsils. Because the size of tonsils in the original image is small and feature extraction is difficult, a two-step detection model based on convolutional neural network is designed in this paper. The first step is to use the oral cavity detection module to centrally detect and cut out oral cavity images from self-made human face data. Step 2, based on the oral image, the position information of tonsils are identified. Then, combined with the depth image, the coordinates of tonsils in the robot world coordinate system are calculated, and the robot arm is controlled to move to the designated position, thus realizing throat swab sampling. Experiments show that the tonsil two-step detection model based on convolutional neural network has high accuracy and detection efficiency, and the AP50 of detection results and the average detection time are better than the contrast algorithm, and the throat swab sampling can be completed accurately.

Key words: target recognition, detection and location, deep learning, medical auxiliary robot

李顺君, 钱强, 史金龙, 葛俊彦, 茅凌波. 基于卷积神经网络的扁桃体咽拭子采样机器人[J]. 计算机工程与应用, 2022, 58(15): 324-329.

LI Shunjun, QIAN Qiang, SHI Jinlong, GE Junyan, MAO Lingbo. Sampling Robot for Tonsil Swabs Based on Convolutional Neural Network[J]. Computer Engineering and Applications, 2022, 58(15): 324-329.

参考文献

[1] LYU Z J，LI Y.Thoughts of the combination of medicine and engineering and collaborative innovation on surgery in China[J].Chinese Journal of Gastrointestinal Surgery，2020，23（6）：562-565.
[2] 谭文君，申存毅，罗禹，等.仿真智能吸痰机器人运动单元的研制[J].中国医疗器械杂志，2019，43（1）：17-20.
TAN W J，SHEN C Y，LUO Y，et al.Development of motion unit of simulated intelligent sputum suction robot[J].Chinese Journal of Medical Instrumentation，2019，43（1）：17-20.
[3] BALTER M L，LEIPHEIMER J M，CHEN A I，et al.Automated end-to-end blood testing at the point-of-care：integration of robotic phlebotomy with downstream sample processing[J].Technology，2018，6（2）：59-66.
[4] YE R Z，ZHOU X L，SHAO F，et al.Feasibility of a 5G-based robot-assisted remote ultrasound system for cardiopulmonary assessment of patients with Coronavirus Disease 2019[J].Chest，2021，159（1）：270-281.
[5] LI S Q，GUO W L，LIU H，et al.Clinical application of an intelligent oropharyngeal swab robot：implication for the COVID-19 pandemic[J].European Respiratory Journal，2020，56（2）：2001912.
[6] RAUTARAY S S，AGRAWAL A.Vision based hand gesture recognition for human computer interaction：a survey[J].Artificial Intelligence Review，2015，43（1）：1-54.
[7] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM，2017，60（6）：84-90.
[8] LECUN Y，BENGIO Y，HINTON G.Deep learning[J].Nature，2015，521（7553）：436-444.
[9] GIRSHICK R，DONAHUE J，DARRELL T，et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition（CVPR）.Columbus，OH，USA：IEEE，2014：580-587.
[10] GIRSHICK R.Fast R-CNN[C]//IEEE International Conference on Computer Vision（ICCV）.Santiago，Chile：IEEE，2015：1440-1448.
[11] LU Y，YU Q，GAO Y，et al.Identification of metastatic lymph nodes in MR imaging with faster region-based convolutional neural networks[J].Cancer Research，2019，78（17）：5135-5143.
[12] LIU J M，WANG D，LU L，et al.Detection and diagnosis of colitis on computed tomography using deep convolutional neural networks[J].Medical Physics，2017，44（9）：4630-4642.
[13] SUN N，YANG D，FANG S，et al.Deep convolutional nets for pulmonary nodule detection and classification[C]//11th International Conference on Knowledge Science，Engineering and Managemet，2018.
[14] DALAL N，TRIGGS B.Histograms of oriented gradients for human detection[C]//Conference on Computer Vision and Pattern Recognition.San Diego，CA，USA：IEEE，2005：886-893.
[15] HE K，GKIOXARI G，DOLLáR P，et al.Mask R-CNN[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，42（2）：386-397.
[16] REN S Q，HE K M，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（6）：1137-1149.
[17] ZHANG Z.A flexiblenew technique for camera calibration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2000，22（11）：1330-1334.
[18] HE K，GKIOXARI G，DOLLáR P，et al.Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2961-2969.