多尺度深层特征加强的CME小样本目标检测模型

doi:10.3778/j.issn.1002-8331.2211-0419

摘要/Abstract

摘要： 针对现有的小样本目标检测模型存在对图像全局语义信息考虑不足、输入图像大小不一而导致检测器性能下降的问题，提出了多尺度深层特征加强的CME小样本目标检测模型。利用大量有标签的基类数据和基于残差跳跃的多层卷积神经网络及多尺度特征增强模块训练一个泛化性良好的模型，经过少量有标签的新类数据和基类数据对模型微调，利用微调后的模型进行目标检测。为验证模型的有效性，使用VOC2007和VOC2012数据集对模型进行训练和评估，相关消融实验证明了引入残差跳跃结构的多层卷积神经网络和多尺度特征增强模块的单独使用和组合使用均可进一步增加模型的准确率。在与6个具有代表性的小样本目标检测模型的对比实验中表明，多尺度深层特征加深的CME比最先进的检测器得分平均提高4.75个百分点。

关键词: 小样本学习, 目标检测, 残差跳跃, 多尺度特征, 深层特征

Abstract: A CME-based few-shot detection model with enhanced multiscale deep feature is proposed to address the problems that existing few-shot detection models have insufficient consideration of global semantic information of images and degradation of detector performance due to varying input image sizes. Firstly, the model is trained with a large amount of labeled base class data and a multilayer convolutional neural network based on residual jumping and a multiscale feature enhanced module with good generalization, then the model is fine-tuned with a small amount of labeled new class data and base class data, and finally the fine-tuned model is used for target detection. To verify the effectiveness of the model, the VOC2007 and VOC2012 datasets are used to train and evaluate the model, and the relevant ablation experiments demonstrate that the introduction of a multilayer convolutional neural network with residual jump structure and the multi-scale feature enhanced module can further increase the accuracy of the model both alone and in combination. In comparison experiments with six representative small-sample target detection models, it is shown that the CME with multiscale deep feature deepening scores outperforms the state-of-the-art detector by an average of 4.75 percentage points.

Key words: few-shot learning, object detection, residual jumping, multiscale feature, deep feature

丁政伟, 白鹤翔, 胡深. 多尺度深层特征加强的CME小样本目标检测模型[J]. 计算机工程与应用, 2024, 60(6): 222-229.

DING Zhengwei, BAI Hexiang, HU Shen. CME-Based Few-Shot Detection Model with Enhanced Multiscale Deep Features[J]. Computer Engineering and Applications, 2024, 60(6): 222-229.

参考文献

[1] REDMON J, DIVVALA S, GIRSHICk R, et al. You only look once: unified, real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 779-788.
[2] REDMON J, FARHADI A. YOLO9000: better, faster, stronger[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 6517-6525.
[3] REDMON J, FARHADI A. YOLOv3: an incremental improvement[J]. arXiv:1804.02767, 2018.
[4] BOCHKOVSKIY A, WANG C Y, LIAO H. YOLOv4: optimal speed and accuracy of object detection[J]. arXiv:2004. 10934, 2020.
[5] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013: 580-587.
[6] GHIASI G, CUI Y, SRINIVAS A, et al. Simple copy-paste is a strong data augmentation method for instance segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 2918-2928.
[7] JIANG S, LIANG S, CHEN C, et al. Class agnostic image common object detection[J]. IEEE Transactions on Image Processing, 2019, 28(6): 2836-2846.
[8] YANG J, LI C, GAO J. Focal modulation networks[J]. arXiv:2203.11926, 2022.
[9] 董博文, 汪荣贵, 杨娟, 等. 结合多尺度特征与掩码图网络的小样本学习[J]. 计算机工程与应用, 2022, 58(16): 111-122.
DONG B W, WANG R G, YANG J, et al. Multi-scale feature enhanced by mask graph neural network for few-shot learning[J]. Computer Engineering and Applications, 2022, 58(16): 111-122.
[10] 张振伟, 郝建国, 黄健, 等. 小样本图像目标检测研究综述[J]. 计算机工程与应用, 2022, 58(5): 1-11.
ZHANG Z W, HAO J G, HUANG J, et al. Review of few-shot object detection[J]. Computer Engineering and Applications, 2022, 58(5): 1-11.
[11] SHEN D G, WU G R, SUK H I. Deep learning in medical image analysis[J]. Annual Review of Biomedical Engineering, 2017, 19: 221-248.
[12] 黄元涛. 基于深度学习的藏羚羊检测与跟踪[D]. 西安: 西安电子科技大学, 2020: 3-69.
HUANG Y T. Detection and tracking of Tibetan antelope based on deep learning[D]. Xi’an: Xidian University, 2020: 3-69.
[13] 徐诚极, 王晓峰, 杨亚东. Attention-YOLO: 引入注意力机制的YOLO检测算法[J]. 计算机工程与应用, 2019, 55(6): 13-23.
XU C J, WANG X F, YANG Y D. Attention-YOLO: YOLO detection algorithm that introduces attention mechanism[J]. Computer Engineering and Applications, 2019, 55(6): 13-23.
[14] CHEN H, WANG Y L, WANG G Y, et al. LSTD: a low-shot transfer detector for object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2018: 2836-2843.
[15] 林润超, 黄荣, 董爱华. 基于注意力机制和元特征二次重加权的小样本目标检测[J]. 计算机应用, 2022, 42(10): 3025-3032.
LIN R C, HUANG R, DONG A H. Few-shot object detection based on attention mechanism and secondary reweighting of meta-features[J]. Journal of Computer Applications, 2022, 42(10): 3025-3032.
[16] 吴晗, 张志龙, 李楚为, 等. 小样本红外图像的样本扩增与目标检测算法[J]. 控制理论与应用, 2021, 38(9): 1477-1485.
WU H, ZHANG Z L, LI C W, et al. Infrared image sample amplification and object detection method with small samples[J]. Control Theory & Applications, 2021, 38(9): 1477-1485.
[17] 杜芸彦, 李鸿, 杨锦辉, 等. 基于负边距损失的小样本目标检测[J]. 计算机应用, 2022, 42(11): 3617-3624.
DU Y Y, LI H, YANG J H, et al. Few-shot target detection based on negative-margin loss[J]. Journal of Computer Applications, 2022, 42(11): 3617-3624.
[18] WANG X, HUANG T E, DARRELL T, et al. Frustratingly simple few-shot object detection[J]. arXiv:2003.06957, 2020.
[19] FAN Q, ZHUO W, TANG C K, et al. Few-shot object detection with attention-RPN and multi-relation detector[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 4013-4022.
[20] SUN B, LI B, CAI S, et al. FSCE: few-shot object detection via contrastive proposal encoding[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021: 7352-7362.
[21] HU H, BAI S, LI A, et al. Dense relation distillation with context-aware aggregation for few-shot object detection[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021: 10185-10194.
[22] AGARWAL A, MAJEE A, SUBRAMANIAN A, et al. Attention guided cosine margin for overcoming class-imbalance in few-shot road object detection[J]. arXiv:2111.06639, 2021.
[23] XIAO Z, QI J, XUE W, et al. Few-shot object detection with self-adaptive attention network for remote sensing images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 4854-4865.
[24] YAN X, CHEN Z, XU A, et al. Meta R-CNN: towards general solver for instance-level few-shot learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 9577-9586.
[25] KANG B, LIU Z, WANG X, et al. Few-shot object detection via feature reweighting[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 8420-8429.
[26] LI B, YANG B, LIU C, et al. Beyond max-margin: class margin equilibrium for few-shot object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021: 7363-7372.
[27] HE K, ZHANG X, REN S. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[28] HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916.
[29] WANG Y X, RAMANAN D, HEBERT M. Meta-learning to detect rare objects[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 9925-9934.