Multi-Scale Feature Enhanced by Mask Graph Neural Network for Few-Shot Learning

doi:10.3778/j.issn.1002-8331.2012-0470

Abstract

Abstract: The ability of extracting sample information determines the effect of the model for few-shot learning（FSL）. To further improve the models’ ability of mining information, a method combining multi-scale feature with mask graph neural network（GNN） is proposed. Firstly, design themini residual network block composed of 1×1 convolution, global average pooling（GAP） and skip connection. Then, stitch mini residual network blocks with the convolutional blocks into feature extractors to extract features of different scales, and use the attention mechanism to fuse multi-scale feature. Secondly, a GNN containing node and edge features is constructed using the fused multi-scale feature. And a meta-learner is added into the GNN to generate edge masks to guide the clustering and updating of nodes by filtering edge features, which further strengthens the sample features. Finally, the computation of class representation is improved through the feature contribution and the pull loss, which enhances the ability of model metric learning. On the MiniImagenet dataset, the 1-shot accuracy of the method is 61.4%, and the 5-shot accuracy is 78.6%, which are 12.0 percentage points and 10.4 percentage points higher than that of the traditional metric learning method, on the Cifar-100 dataset, increases of 9.7 percentage points and 6.0 percentage points are reached respectively. Therefore, the method improves significantly the accuracy of few-shot learning based on metric learning.

Key words: few-shot learning, metric learning, meta-learning, multi-scale feature, graph neural network

摘要： 对样本所含信息的提取能力决定网络模型进行小样本分类的效果，为了进一步提高模型挖掘信息的能力，提出一种结合多尺度特征与掩码图网络的小样本学习方法。设计由1×1卷积、全局平均池化和跳跃连接组成的最小残差神经网络块，与卷积块拼接成特征提取器，以提取样本不同尺度的特征，并通过注意力机制将不同尺度特征融合；使用融合的多尺度特征构建包含结点与边特征的图神经网络，并在其中加入一个元学习器（meta-learner）用于生成边的掩码，通过筛选边特征来指导图结点聚类与更新，进一步强化样本特征；通过特征贡献度和互斥损失改进类在嵌入空间表达特征的求解过程，提升模型度量学习能力。在MiniImagenet数据集上，该方法1-shot准确率为61.4%，5-shot准确率为78.6%，分别超过传统度量学习方法12.0个百分点与10.4个百分点；在Cifar-100数据集上分别提升9.7个百分点和6.0个百分点。该方法有效提升了小样本学习场景下的模型分类准确率。

关键词: 小样本学习, 度量学习, 元学习, 多尺度特征, 图神经网络

DONG Bowen, WANG Ronggui, YANG Juan, XUE Lixia. Multi-Scale Feature Enhanced by Mask Graph Neural Network for Few-Shot Learning[J]. Computer Engineering and Applications, 2022, 58(16): 111-122.

董博文, 汪荣贵, 杨娟, 薛丽霞. 结合多尺度特征与掩码图网络的小样本学习[J]. 计算机工程与应用, 2022, 58(16): 111-122.

References

[1] GIRSHICK R，DONAHUE J，DARRELL T，et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition，Columbus，2014：580-587.
[2] GIRSHICK R.Fast R-CNN[C]//Proceedings of 2015 IEEE International Conference on Computer Vision，Santiago，2015：1440-1448.
[3] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2016，39（6）：1137-1149.
[4] SZEGEDY C，LIU W，JIA Y，et al.Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition，Boston，2015：1-9.
[5] MIKOLOV T，SUTSKEVER I，CHEN K，et al.Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 27th Neural Information Processing Systems.Lake Tahoe：Curran Associates，Inc，2013：3111-3119.
[6] PETERS M E，NEUMANN M，IYYER M，et al.Deep contextualized word representations[EB/OL].[2020-12-14].https：//arxiv.org/pdf/1802.05365.pdf.
[7] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition，Las Vegas，2016：770-778.
[8] FINK M.Object classification from a single example utilizing class relevance metrics[C]//Proceedings of the 18th Conference on Neural Information Processing Systems，Vancouver，2004：449-456.
[9] FEI-FEI L，FERGUS R，PERONA P.One-shot learning of object categories[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2006，28（4）：594-611.
[10] TREMBLAY J，PRAKASH A，ACUNA D，et al.Training deep networks with synthetic data：bridging the reality gap by domain randomization[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition Workshops，Salt Lake City，2018：969-977.
[11] HARIHARAN B，GIRSHICK R.Low-shot visual recognition by shrinking and hallucinating features[C]//Proceedings of 2017 IEEE International Conference on Computer Vision，Venice，2017：3018-3027.
[12] WANG P，LIU L，SHEN C，et al.Multi-attention network for one shot learning[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，2017：2721-2729.
[13] MISHRA N，ROHANINEJAD M，CHEN X，et al.A simple neural attentive meta-learner[EB/OL].（2017-07-11）[2020-12-14].https：//arxiv.org/pdf/1707.03141.pdf.
[14] MUNKHDALAI T，YU H.Meta networks[C]//Proceedings of the 34th International Conference on Machine Learning，Sydney，2017：2554-2563.
[15] FINN C，ABBEEL P，LEVINE S.Model-agnostic meta-learning for fast adaptation of deep networks[C]//Proceedings of the 34th International Conference on Machine Learning，Sydney，2017：1126-1135.
[16] KOCH G，ZEMEL R，SALAKHUTDINOV R.Siamese neural networks for one-shot image recognition[C]//ICML Deep Learning Workshop，2015，2.
[17] VINYALS O，BLUNDELL C，LILLICRAP T，et al.Matching networks for one shot learning[C]//Proceedings of the 30th Conference on Neural Information Processing Systems，Barcelona，2016：3630-3638.
[18] SNELL J，SWERSKY K，ZEMEL R.Prototypical networks for few-shot learning[C]//Proceedings of the 31st Conference on Neural Information Processing Systems，Long Beach，2017：4077-4087.
[19] LIU W，ANGUELOV D，ERHAN D，et al.SSD：single shot multibox detector[C]//Proceedings of the 14th European Conference on Computer Vision.Cham：Springer，2016：21-37.
[20] LIN T Y，DOLLáR P，GIRSHICK R，et al.Feature pyramid networks for object detection[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，2017：2117-2125.
[21] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems，Long Beach，2017：5998-6008.
[22] ZAMIR A R，SAX A，SHEN W，et al.Taskonomy：disentangling task transfer learning[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake City，2018：3712-3722.
[23] YU M，GUO X，YI J，et al.Diverse few-shot text classification with multiple metrics[EB/OL].（2018-05-19）[2020-12-14].https：//arxiv.org/pdf/1805.07513.pdf.
[24] CHEN Z，FU Y，ZHANG Y，et al.Multi-level semantic feature augmentation for one-shot learning[J].IEEE Transactions on Image Processing，2019，28（9）：4594-4605.
[25] ALFASSY A，KARLINSKY L，AIDES A，et al.LASO：label-set operations networks for multi-label few-shot learning[C]//Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，2019：6548-6557.
[26] WANG X，YU F，WANG R，et al.TAFE-Net：task-aware feature embeddings for low shot learning[C]//Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，2019：1831-1840.
[27] RAVI S，LAROCHELLE H.Optimization as a model for few-shot learning[C]//Proceedings of the 5th International Conference on Learning Representations，2017.
[28] SUN Q，LIU Y，CHUA T S，et al.Meta-transfer learning for few-shot learning[C]//Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，2019：403-412.
[29] KESHARI R，VATSA M，SINGH R，et al.Learning structure and strength of CNN filters for small sample size training[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake City，2018：9349-9358.
[30] BONEY R，ILIN A.2018.Semi-supervised few-shot learning with maml[C]//Proceedings of the 6th International Conference on Learning Representations，2018.
[31] JAMAL M A，QI G J.Task agnostic meta-learning for few-shot learning[C]//Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，2019：11719-11727.
[32] XING E P，JORDAN M I，RUSSELL S J，et al.Distance metric learning with application to clustering with side-information[C]//Proceedings of the 16th Conference on Neural Information Processing Systems，Vancouver，2002：521-528.
[33] SUNG F，YANG Y，ZHANG L，et al.Learning to compare：relation network for few-shot learning[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake City，2018：1199-1208.
[34] LIU Y，LEE J，PARK M，et al.Learning to propagate labels：transductive propagation network for few-shot learning[EB/OL].（2018-05-25）[2020-12-14].https：//arxiv.org/pdf/1805.10002.pdf.
[35] KIM J，KIM T，KIM S，et al.Edge-labeling graph neural network for few-shot learning[C]//Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition，Long Beach，2019：11-20.
[36] LIN M，CHEN Q，YAN S.Network in network[EB/OL].（2013-12-16）[2020-12-14].https：//arxiv.org/pdf/1312.4400.pdf.
[37] GORI M，MONFARDINI G，SCARSELLI F.A new model for learning in graph domains[C]//Proceedings of 2005 IEEE International Joint Conference on Neural Networks，Montreal，2005，2：729-734.
[38] BANERJEE A，MERUGU S，DHILLON I S，et al.Clustering with Bregman Divergences[J].Journal of Machine Learning Research，2005，6（4）：1705-1749.
[39] QIAO L，SHI Y，LI J，et al.Transductive episodic-wise adaptive metric for few-shot learning[C]//Proceedings of 2019 IEEE International Conference on Computer Vision，Seoul，2019：3603-3612.
[40] MAATEN L，HINTON G.Visualizing data using t-SNE[J].Journal of Machine Learning Research，2008，9：2579-2605.
[41] YE H J，HU H，ZHAN D C，et al.Few-shot learning via embedding adaptation with set-to-set functions[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition，Seattle，2020：8808-8817.
[42] CHEN M，FANG Y，WANG X，et al.Diversity transfer network for few-shot learning[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence，New York，2020：10559-10566.