Multilevel Metric Networks for Few-Shot Learning

doi:10.3778/j.issn.1002-8331.2107-0351

Abstract

Abstract: The classification results of few-shot learning depend on the model’s ability to express the sample features. In order to further mine the semantic information expressed by images, a multilevel metric networks few-shot learning method is proposed. Firstly, the feature vector of the input image is put into the embedded module for feature extraction. Secondly, the feature descriptors obtained through the second layer convolution and the third layer convolution are measured by image-class to obtain the score of image relation respectively. The feature vectors obtained by the fourth layer convolution are fully connected and used as the image-image metric to obtain the image dependency probability. Finally, the weighted fusion of two image relational scores and one image membership probability is performed through cross validation and the classification results are output. The experimental results show that the accuracy of 5-way 1-shot and 5-way 5-shot of the proposed method is 56.77% and 75.83% in the miniImageNet dataset. The accuracy of 5-way 1-shot and 5-way 5-shot increases to 55.34% and 76.32%, respectively, on CUB dataset. The accuracy of the proposed method on Omniglot dataset is also improved compared with the traditional method. Therefore, this method can effectively mine the semantic information expressed in the image, and significantly improve the accuracy of few-shot image classification.

Key words: few-shot learning, metric learning, deep learning, multistage measure

摘要： 小样本学习的分类结果依赖于模型对样本特征的表达能力，为了进一步挖掘图像所表达的语义信息，提出一种多级度量网络的小样本学习方法。将输入图像的特征向量放入嵌入模块进行特征提取；将经过第二层卷积及第三层卷积得到的特征描述子分别进行图像-类的度量以获得图像关系得分，对第四层卷积得到的特征向量进行全连接并将其做图像-图像的度量从而得到图像从属概率；通过交叉验证对2个图像关系得分以及1个图像从属概率进行加权融合并输出分类结果。实验结果表明在miniImageNet数据集上，该方法5-way 1-shot准确率为56.77%，5-way 5-shot准确率为75.83%。在CUB数据集上，该方法5-way 1-shot及5-way 5-shot准确率分别上升到55.34%及76.32%。在Omniglot数据集上准确率同传统方法相比也有一定提升。因此，该方法可有效挖掘图像中所表达的语义信息，显著提高小样本图像分类的准确率。

关键词: 小样本学习, 度量学习, 深度学习, 多级度量

WEI Shihong, LIU Hongmei, TANG Hong, ZHU Longjiao. Multilevel Metric Networks for Few-Shot Learning[J]. Computer Engineering and Applications, 2023, 59(2): 94-101.

韦世红, 刘红梅, 唐宏, 朱龙娇. 多级度量网络的小样本学习[J]. 计算机工程与应用, 2023, 59(2): 94-101.

References

[1] SANTORO A，BARTUNOV S，BOTVINICK M，et al.Meta-learning with memory-augmented neural networks[C]//Proceedings of the International Conference on Machine Learning.New York：ACM，2016：1842-1850.
[2] GOODFELLOW I，POUGET A J，MIRZA M，et al.Generative adversarial nets[C]//Proceedings of Conference and Workshop on Neural Information Processing Systems，Vancouver，Canada，2014：2672-2680.
[3] GERG I D，WILLIAMS D P，MONGA V.Data adaptive image enhancement and classification for synthetic aperture sonar[C]//2020 IEEE International Geoscience and Remote Sensing Symposium，2020：2835-2838.
[4] PENG Z，LI Z，ZHANG J，et al.Few-shot image recognition with knowledge transfer[C]//2019 IEEE/CVF International Conference on Computer Vision（ICCV），2019：441-449.
[5] CHEN P，LI P，LI Q，et al.Semi-supervised fine-grained image categorization using transfer learning with hierarchical multi-scale adversarial networks[J].IEEE Access，2019，7：118650-118668.
[6] FINN C，ABBEEL P，LEVINE S.Model-agnostic meta-learning for fast adaptation of deep networks[C]//Proceedings of the 34th International Conference on Machine Learning，2017：1126-1135.
[7] CAO Z.Meta-Seg：a generalized meta-learning framework for multi-class few-shot semantic segmentation[J].IEEE Access，2019，7：166109-166121.
[8] VINYALS O，BLUNDELL C，LILLICRAP T，et al.Matching networks for one shot learning[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems.Barcelona，Spain：MIT Press，2016：3630-3638.
[9] SNELL J，SWERSKY K，ZEMEL R.Prototypical networks for few-shot learning[C]//Advances in Neural Information Processing Systems，2017：4077-4087.
[10] SUNG F，YANG Y X，ZHANG L，et al.Learning to compare：relation network for few-shot learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake，USA，2018：1199-1208.
[11] CAI Q，PAN Y W，YAO T，et al.Memory matching networks for one-shot image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake City，Utah，2018：4080-4088.
[12] PAN Y，YAO T，LI Y，et al.Transferrable prototypical networks for unsupervised domain adaptation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2019：2234-2242.
[13] HUI B，ZHU P，HU Q，et al.Self-attention relation network for few-shot learning[C]//2019 IEEE International Conference on Multimedia & Expo Workshops（ICMEW），2019：198-203.
[14] BOIMAN O，SHECHTMAN E，IRANI M.Indeference of nearest-neighbor based image classification[C]//IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2008：1-8.
[15] LI W，WANG L，XU J，et al.Revisiting local descriptor based image-to-class measure for few-shot learning[J].arXiv：1903.12290，2019.
[16] 徐传运，孙越，李刚，等.基于深度度量学习的小样本商品图像分类研究[J].重庆理工大学学报（自然科学），2020，34（9）：209-216.
XU C Y，SUN Y，LI G，et al.Few-shot retail product image classification based on deep metric learning[J].Journal of Chongqing University of Technology（Natural Science），2020，34（9）：209-216.
[17] LIN T Y，ROY C A，MAJI S.Bilinear CNN models for fine-grained visual recognition[C]//Proceedings of the 15th IEEE International Conference on Computer Vision，Santiago，2015：1449-1457.
[18] LAKE B M，SALAKHUTDINOV R，TENENBAUM J B.Human-level concept learning through probabilistic program induction[J].Science，2015，350（6266）：1332-1338.
[19] DENG J，DONG W，SOCHER R，et al.ImageNet：a large-scale hierarchical image database[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2009：20-25.
[20] WAH C，BRANSON S，WELINDER P，et al.The Caltech-UCSD Birds-200-2011 Dataset[Z].2011.

[21] SASHANK J R，SATYEN K，SANJIV K.On the convergence of adam and beyond[C]//Proceedings of the Internatiomal Conference on Learning Representations（ICLR），2018.

[22] ZAHEER R，SHAZIYA H.A study of the optimization algorithms in deep learning[C]//2019 Third International Conference on Inventive Systems and Control（ICISC），2019：536-539.