Memory-Based Transfer Learning for Few-Shot Learning

doi:10.3778/j.issn.1002-8331.2102-0259

Abstract

Abstract: Few-shot learning is an important field in visual recognition which aims to learn new visual concepts with limited data. To address this challenge, some meta-learning methods propose to extract transferable knowledge from auxiliary tasks and apply the knowledge to the target task. In order to transfer knowledge better, a memory-based transfer learning method is proposed. Firstly, weights decomposition strategy is proposed to decompose the weights into the frozen weights and the learnable weights. In transfer learning, by fixing the frozen weights and updating only the learnable weights, the parameters that the model needs to learn can be reduced. Next, an additional memory module is used to store the experience of previous tasks. When learning novel tasks, the experience can be used to initialize the parameter states of the model for better transfer learning. According to experimental results on miniImageNet, tieredImageNet and CUB datasets, compared with state-of-the-art methods, the proposed method achieves competitive or even better performance for few-shot classification.

Key words: few-shot learning, transfer learning, memory module, meta-learning

摘要： 小样本学习是视觉识别中的一个受关注的领域，旨在通过少量的数据来学习新的视觉概念。为了解决小样本问题，一些元学习方法提出从大量辅助任务中学习可迁移的知识并将其应用于目标任务上。为了更好地对知识进行迁移，提出了一种基于记忆的迁移学习方法。提出一种权重分解策略，将部分权重分解为冻结权重与可学习权重，在迁移学习中通过固定冻结权重，仅更新可学习权重的方式来减少模型需要学习的参数。通过一个额外的记忆模块来存储之前任务的经验，在学习新任务时，这些经验被用来初始化模型的参数状态，以此更好地进行迁移学习。通过在miniImageNet、tieredImageNet以及CUB数据集上的实验结果表明，相对于其他先进的方法，该方法在小样本分类任务上取得了具有竞争力甚至是更好的表现。

关键词: 小样本学习, 迁移学习, 记忆模块, 元学习

LIU Bing, YANG Juan, WANG Ronggui, XUE Lixia. Memory-Based Transfer Learning for Few-Shot Learning[J]. Computer Engineering and Applications, 2022, 58(19): 242-249.

刘兵, 杨娟, 汪荣贵, 薛丽霞. 结合记忆与迁移学习的小样本学习[J]. 计算机工程与应用, 2022, 58(19): 242-249.

References

[1] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[2] SZEGEDY C，VANHOUCKE V，IOFFE S，et al.Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：2818-2826.
[3] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems，2012：1097-1105.
[4] HE K，GKIOXARI G，DOLLáR P，et al.Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2961-2969.
[5] REDMON J，DIVVALA S，GIRSHICK R，et al.You only look once：Unified，real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：779-788.
[6] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：Towards real-time object detection with region proposal networks[J].arXiv：1506.01497，2015.
[7] BAHDANAU D，CHO K，BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv：1409.0473，2014.
[8] WU Y，SCHUSTER M，CHEN Z，et al.Google’s neural machine translation system：Bridging the gap between human and machine translation[J].arXiv：1609.08144，2016.
[9] CHEN Z，FU Y，ZHANG Y，et al.Multi-level semantic feature augmentation for one-shot learning[J].IEEE Transactions on Image Processing，2019，28（9）：4594-4605.
[10] FINN C，ABBEEL P，LEVINE S.Model-agnostic metalearning for fast adaptation of deep networks[C]//Proceedings of the 34th International Conference on Machine Learning，2017：1126-1135
[11] ZHANG H，ZHANG J，KONIUSZ P.Few-shot learning via saliency-guided hallucination of samples[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：2770-2779.
[12] WANG Y X，GIRSHICK R，HEBERT M，et al.Low-shot learning from imaginary data[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7278-7286.
[13] KOCH G，ZEMEL R，SALAKHUTDINOV R.Siamese neural networks for one-shot image recognition[C]//Proceedings of ICML Deep Learning Workshop，2015.
[14] VINYALS O，BLUNDELL C，LILLICRAP T，et al.Matching networks for one shot learning[C]//Proceedings of the 30th Conference on Neural Information Processing Systems，2016：3630-3638.
[15] SNELL J，SWERSKY K，ZEMEL R.Prototypical networks for few-shot learning[C]//Proceedings of the 31st Conference on Neural Information Processing Systems，2017：4077-4087.
[16] SUNG F，YANG Y，ZHANG L，et al.Learning to compare：Relation network for few-shot learning[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：1199-1208.
[17] LI W，WANG L，XU J，et al.Revisiting local descriptor based image-to-class measure for few-shot learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：7260-7268.
[18] LI W，XU J，HUO J，et al.Distribution consistency based covariance metric networks for few-shot learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2019：8642-8649.
[19] RAVI S，LAROCHELLE H.Optimization as a model for few-shot learning[C]//Proceedings of the International Conference on Learning Representations，2017.
[20] LI Z，ZHOU F，CHEN F，et al.Meta-SGD：Learning to learn quickly for few-shot learning[J].arXiv：1707.09835，2017.
[21] LIU Y，LEE J，PARK M，et al.Learning to propagate labels：Transductive propagation network for few-shot learning[EB/OL].（2018-05-25）[2020-12-14].https：//arxiv.org/pdf/1805.10002.pdf.
[22] YE H J，HU H，ZHAN D C，et al.Few-shot learning via embedding adaptation with set-to-set functions[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：8808-8817.
[23] LI S，CHEN D，LIU B，et al.Memory-based neighbourhood embedding for visual recognition[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：6102-6111.
[24] SANTORO A，BARTUNOV S，BOTVINICK M，et al.Meta-learning with memory-augmented neural networks[C]//Proceedings of the International Conference on Machine Learning，2016：1842-1850.
[25] HE J，HONG R，LIU X，et al.Memory-augmented relation network for few-shot learning[C]//Proceedings of the 28th ACM International Conference on Multimedia，2020：1236-1244.
[26] RUSSAKOVSKY O，DENG J，SU H，et al.Imagenet large scale visual recognition challenge[J].International Journal of Computer Vision，2015，115（3）：211-252.
[27] REN M，TRIANTAFILLOU E，RAVI S，et al.Meta-learning for semi-supervised few-shot classification[J].arXiv：1803. 00676，2018.
[28] WAH C，BRANSON S，WELINDER P，et al.The caltech-ucsd birds-200-2011 dataset[Z].2011.
[29] HILLIARD N，PHILLIPS L，HOWLAND S，et al.Few-shot learning with metric-agnostic conditional embeddings[J].arXiv：1802.04376，2018.
[30] IOFFE S，SZEGEDY C.Batch normalization：Accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the International Conference on Machine Learning，2015：448-456.
[31] KINGMA D P，BA J.Adam：A method for stochastic optimization[J].arXiv：1412.6980，2014.