Survey of Few-Shot Image Classification Based on Deep Meta-Learning

doi:10.3778/j.issn.1002-8331.2308-0271

Abstract

Abstract: Deep meta-learning has emerged as a popular paradigm for addressing few-shot classification problems. A comprehensive review of recent advancements in few-shot image classification algorithms based on deep meta-learning is provided. Starting from the problem description, the categorizes of the algorithms based on deep meta-learning for few-shot image classification are summarized, and commonly used few-shot image classification datasets and evaluation criteria are introduced. Subsequently, typical models and the latest research progress are elaborated in three aspects: model-based deep meta-learning methods, optimization-based deep meta-learning methods, and metric-based deep meta-learning methods. Finally, the performance analysis of existing algorithms on popular public datasets is presented, the research hotspots in this topic are summarized, and its future research directions are discussed.

Key words: deep learning, meta learning, few-shot learning, image classification

摘要： 深度元学习是解决小样本分类问题的流行范式。对近年来基于深度元学习的小样本图像分类算法进行了详细综述。从问题的描述出发对基于深度元学习的小样本图像分类算法进行概括，并介绍了常用小样本图像分类数据集及评价准则；分别从基于模型的深度元学习方法、基于优化的深度元学习方法以及基于度量的深度元学习方法三个方面对其中的典型模型以及最新研究进展进行详细阐述。最后，给出了现有算法在常用公开数据集上的性能表现，总结了该课题中的研究热点，并讨论了未来的研究方向。

关键词: 深度学习, 元学习, 小样本学习, 图像分类

ZHOU Bojun, CHEN Zhiyu. Survey of Few-Shot Image Classification Based on Deep Meta-Learning[J]. Computer Engineering and Applications, 2024, 60(8): 1-15.

周伯俊, 陈峙宇. 基于深度元学习的小样本图像分类研究综述[J]. 计算机工程与应用, 2024, 60(8): 1-15.

References

[1] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 26th Annual Conference?on Neural Information Processing Systems (NIPS’12), 2012: 1097-1105.
[2] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521: 436-444.
[3] HE K, ZHANG X, REN S. Deep residual learning for image recognition[C]//Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16), 2016: 770-778.
[4] LAKE B M, SALAKHUTDINOV R, GROSS J, et al. One shot learning of simple visual concepts[J]. Cognitive Science Society, 2011, 33: 2568-2573.
[5] DSD D, LEE C S G. A two-stage approach to few-shot learning for image recognition[J]. IEEE Transactions on Image Processing, 2020, 29(12): 3336-3350.
[6] ZHOU B J, ZHAO J H, YAN C K, et al. Global and local knowledge distillation method for few-shot classification of electrical equipment[J]. Applied Science, 2023, 13(12): 7016-7028.
[7] 黄文东. 基于元学习的小样本遥感图像分类研究[D]. 重庆: 重庆邮电大学, 2022.
HUANG W D. Research on few-shot remote sensing image classification based on meta-learning[D]. Chongqing: Chongqing University of Posts and Telecommunications, 2022.
[8] FAN Q, ZHUO W, TANG C K, et al. Few-shot object detection with attention-RPN and multi-relation detector[C]//Proceedings of the 33rd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020: 4012-4021.
[9] 刘春磊, 陈天恩, 王聪, 等. 小样本目标检测研究综述[J]. 计算机科学与探索, 2023, 17(1): 53-73.
LIU C L, CHEN T E, WANG C, et al. Survey of few-shot object detection[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(1): 53-73.
[10] HUTTER F, KOTTHOFF L, VANSCHOREN J. Automated machine learning: methods, systems, challenges[M]. [S.l.]：Springer Publishing Company, Incorporated, 2019.
[11] ZHOU L, CUI P, JIA X, et al. Learning to select base classes for few-shot classification[C]//Proceedings of the 33rd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020: 4623-4632.
[12] ZHANG C, LI C, CHENG J. Few-shot visual classification using image pairs with binary transformation[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(9): 2867-2871.
[13] WANG Y Q, YAO Q M, KWOK J T, et al. Generalizing from a few examples: a survey on few-shot learning[J]. ACM Computing Surveys, 2020, 53(3): 1-34.
[14] 刘颖, 雷研博, 范九伦, 等. 基于小样本学习的图像分类技术综述[J]. 自动化学报, 2021, 47(2): 297-315.
LIU?Y, ?LEI?Y B, ?FAN?J L, ?et al. Survey on image classification technology based on small sample learning[J]. Acta Automatica?Sinica, ?2021, 47(2): 297-315.
[15] 赵凯琳, 靳小龙, 王元卓. 小样本学习综述[J]. 软件学报, 2021, 32(2): 349-369.
ZHAO K L, JIN X L, WANG Y Z. Survey on few-shot learning[J]. Journal of Software, 2021, 32(2): 349-369.
[16] 祝钧桃, 姚光乐, 张葛祥, 等. 深度神经网络的小样本学习综述[J]. 计算机工程与应用, 2021, 57(7): 22-33.
ZHU J T, YAO G L, ZHANG G X, et al. Survey of few shot learning of deep neural network[J]. Computer Engineering and Applications, 2021, 57(7): 22-33.
[17] 胡西范, 陈世平. 基于机器学习的小样本学习综述[J]. 智能计算机与应用, 2021, 11(7): 191-195.
HU X F, CHEN S P. A survey of few-shot learning based on machine learning[J]. Intelligent Computer and Applications, 2021, 11(7): 191-195.
[18] 葛轶洲, 刘恒, 王言, 等. 小样本困境下的深度学习图像识别综述[J]. 软件学报, 2022, 33(1): 193-210.
GE Y Z, LIU H, WANG Y, et al. Survey on deep learning image recognition in dilemma of small samples[J]. Journal of Software, 2022, 33(1): 193-210.
[19] 彭云聪, 秦小林, 张力戈, 等. 面向图像分类的小样本学习算法综述[J]. 计算机科学, 2022, 49(5): 1-9.
PENG Y C, QIN X L, ZHANG L G, et al. Survey on few-shot learning algorithms for image classification[J]. Computer Science, 2022, 49(5): 1-9.
[20] 陈良臣, 傅德印. 面向小样本数据的机器学习方法研究综述[J]. 计算机工程, 2022, 48(11): 1-13.
CHEN L C, FU D Y. Survey on machine learning methods for small sample data[J]. Computer Engineering, 2022, 48(11): 1-13.
[21] 安胜彪, 郭昱岐, 白宇, 等. 小样本图像分类研究综述[J]. 计算机科学与探索, 2023, 17(3): 511-532.
AN S B, GUO Y Q, BAI Y, et al. Survey of few-shot image classification research[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(3): 511-532.
[22] 潘雪玲, 李国和, 郑艺峰. 面向深度网络的小样本学习综述[J]. 计算机应用研究, 2023, 40(10): 2881-2888.
PAN X L, LI G H, ZHENG Y F. Survey on few-shot learning for deep network[J]. Application Research of Computers, 2023, 40(10): 2881-2888.
[23] 罗建豪, 吴建鑫. 基于深度卷积特征的细粒度图像分类研究综述[J]. 自动化学报, 2017, 43(8): 1306-1318.
LUO J H, WU J X. A survey on fine-grained image categorization using deep convolutional features[J]. Acta Automatica?Sinica, 2017, 43(8): 1306?1318.
[24] 刘鑫, 周凯锐, 何玉琳, 等. 基于度量的小样本分类方法研究综述[J]. 模式识别与人工智能, 2021, 34(10): 909-923.
LIU X, ZHOU K R, HE Y L, et al. Survey of metric-based few-shot classification[J]. Pattern Recognition and Artificial Intelligence, 2021, 34(10): 909-923.
[25] LIU W, ZHANG C, LIN G, et al. CRNet: cross-reference networks for few-shot segmentation[C]//Proceedings of the 33rd IEEE Conference on Computer Vision and Pattern Recognition(CVPR’20), 2020: 4164-4172.
[26] SCHMIDHUBER J. Evolutionary principles in self-refe- rential learning[D]. Munich: Technical University of Munich, 1987.
[27] HINTON G E, PLAUT D C. Using fast weights to deblur old memories[C]//Proceedings of the 9th Annual Conference of the Cognitive Science Society, 1987: 177-186.
[28] VILALTA R, DRISSI Y. A perspective view and survey of meta-learning[J]. Artificial Intelligence Review, 2002, 18(2): 77-95.
[29] MIKE H, JAN N R, ASKE P. A survey of deep me-ta-learning[J]. Artificial Intelligence Review, 2021, 54(6): 4483-4541.
[30] HOSPEDALES T, ANTONIOU A, MICAELLI P, et al. Meta-learning in neural networks: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(9): 5149-5169.
[31] VINYALS O, BLUNDELL C, LILLICAP T, et al. Matching networks for one shot learning[C]//Proceedings of the 30th Annual Conference?on Neural Information Processing Systems (NIPS’16), 2016: 3637-3645.
[32] SANTORO A, BARTUNOV S, BOTVINICK M. One-shot learning with memory-augmented neural networks [C]//Proceedings of the 33rd International Conference on Machine learning (ICML’16), 2016: 1842-1850.
[33] MISHRA N, ROHANINEJAD M, CHEN X, et al. A simple neural attentive meta-learner[J]. arXiv：1707.03141, 2017.
[34] MUNKHDALAI T Y H. Meta networks[C]//Proceedings of the 34th International Conference on Machine Learning (ICML’17), 2017: 2554-2563.
[35] FINN C, ABBEEL P, LEVINE S. ?Model-agnostic meta- learning for fast adaptation of deep network[C]//Proceedings of the 34th International Conference on Machine Learning(ICML’17), 2017: 1126-1135.
[36] RAVI S, LAROCHELLE H. Optimization as a model for few-shot learning[C]//Proceedings of 5th International Conference on Learning Representations (ICLR’17), 2017: 1-17.
[37] NICHOL A, ACHIAM J, SCHULMAN J. Reptile: on first-order meta-learning algorithms[J]. arXiv: 1803. 02999, 2018.
[38] RAJESWARAN A, ?FINN C, KAKADE S, et al. Meta learning with implicit gradient[C]//Proceedings of the 32nd Annual Conference?on Neural Information Processing Systems(NIPS’18), 2018: 113-124.
[39] RUSU A, RAO D, SYGNOWSKI J. Meta-learning with latent embedding optimization[C]//Proceedings of 6th International Conference on Learning Representations (ICLR’18), 2018: 1-17.
[40] BAIK S, HONG S, LEE K M. Learning to forget for meta-learning[C]//Proceedings of the 33rd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020: 2376-2384.
[41] SNELL J, SWERRSKY K, ZEMEL R. Prototypical networks for few-shot learning[C]//Proceedings of the 31st Annual Conference?on Neural Information Processing Systems (NIPS’17), 2017: 4077-4087.
[42] SUNG F, YANG Y, ZHANG L, et al. Learning to compare: relation network for few-shot learning[C]//Proceedings of the 31st IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18), 2018: 1199-1208.
[43] HOU R B, CHANG H, MA B P. Cross attention network for few-shot classification[C]//Proceedings of the 33rd Annual Conference?on Neural Information Processing Systems (NIPS’19), 2019: 4003-4014.
[44] SIMON C, KONOUSZ P, NOCK R, et al. Adaptive sub-spaces for few-shot learning[C]//Proceedings of the 33rd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020: 4135-4144.
[45] LI A, HUANG W, LAN X, et al. Boosting few-shot learning with adaptive margin loss[C]//Proceedings of the 33rd IEEE Conference on Computer Vision and Pattern Recognition(CVPR’20), 2020: 12573-12581.
[46] LI W, WANG L, XU J, et al. Revisiting local descriptor-based image-to-class measure for few-shot learning[C]//Proceedings of the 32nd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19), 2019: 7260-7268.
[47] ZHANG C, CAI Y, LIN G, et al. DeepEMD: few-shot image classification with differentiable Earth Mover’s distance and structured classifiers[C]//Proceedings of the 33rd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020: 12200-12210.
[48] CHEN Y B, WANG X L, LIU Z, et al. A new meta-baseline for few-shot learning[C]//Proceedings of?the 34th AAAI Conference on Artificial Intelligence(AAAI’20), 2020: 1-8.
[49] LIU Y, ZHANG W F, XIANG C, et al. Learning to affiliate: mutual centralized learning for few-shot classification[C]//Proceedings of the 35rd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’22), 2022: 14391-14400.
[50] LAKE B M, SALAKHUTDINOV R, TENENBAUM J B. Human-level concept learning through probabilistic program induction[J]. Science, 2015, 350: 1332-1338.
[51] RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252.
[52] REN M, TRIANTAFILLOU E, RAVI S, et al. Meta-learning for semi-supervised few-shot classification[C]//Proceedings of the 6th International Conference on Learning Representations (ICLR’18), 2018: 1-15.
[53] ORESHKIN B, RODRIGUEZ P, LACOSTE A. TADAM: task dependent adaptive metric for improved few-shot learning[C]//Proceedings of the 32nd Annual Conference?on Neural Information Processing Systems (NIPS’18), 2018: 719-729.
[54] YE H J, HU H X, ZHAN D C, et al. Learning embedding adaptation for few-shot learning[J]. arXiv:1812.03664, 2018.
[55] 董安国, 张倩, 刘洪超, 等. 基于TSNE和多尺度稀疏自编码的高光谱图像分类[J]. 计算机工程与应用, 2019, 55（21）：177-182.
DONG A G, ZHANG Q, LIU H C, et al. Hyperspectral image classification based on TSNE and multiscale sparse auto-encoder[J]. Computer Engineering and Applications, 2019, 55(21): 177-182.
[56] GARNELO M, ROSENBAUM D, MADDISON C J. Conditional neural processes[C]//Proceedings of the 35th International Conference on Machine Learning (ICML’18), 2018: 1704-1713.
[57] 段港海. 结合小样本指导的元学习图像分类算法研究[D]. 长春: 吉林大学, 2023.
DUAN G H. Few-shot directed meta-learning for image classification[D]. Changchun: Jilin University, 2023.
[58] 刘杰豪. 基于深度判别性特征学习的少样本图像分类算法研究[D]. 广州: 广州大学, 2022.
LIU J H. Research on few-shot image classification algorithm based on deep discriminative feature learning[D]. Guangzhou: Guangzhou University, 2022.
[59] LI Z G, ZHOU F W, CHEN F, et al. Meta-SGD: learning to learn quickly for few-shot learning[J]. arXiv:1707.09835, 2017.
[60] PARK E, OLIVA J B. Meta-curvature[C]//Proceedings of the 33rd Annual Conference?on Neural Information Processing Systems (NIPS’19), 2019: 3314-3324.
[61] SONG X Y, ?GAO W B, ?YANG Y X, ?et al. ES-MAML: simple Hessian-free meta learning[C]//Proceedings of the 8th International Conference on Learning Representations (ICLR’20), 2020: 1-22.
[62] SUN Q, LIU Y, CHUA T, et al. Meta-transfer learning for few-shot learning[C]//Proceedings of the 32nd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19), 2019: 403-412.
[63] JAMAL M A, ?QI G J, ?SHAH M. Task agnostic meta-learning for few-shot learning[C]//Proceedings of the 32nd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19), 2019: 11711-11719.
[64] TIAN H D, LIU B, YUAN X T. Meta-learning with network pruning[C]//Proceedings of the 16th European Conference on Computer Vision (ECCV’20), 2020: 675-700.
[65] VUORIO R, ?SUN S H, ?HU H X. Multimodal model-agnostic meta-learning via task-aware modulation[C]//Proceedings of the 33rd Annual Conference?on Neural Information Processing Systems (NIPS’19), 2019: 1-12.
[66] LEE H B, LEE H, NA D, et al. Learning to balance: bayesian meta-learning for imbalanced and out-of-distribution tasks[C]//Proceedings of the 8th International Conference on Learning Representations (ICLR’20), 2020: 1-15.
[67] FINN C, XU K, LEVINE S. Probabilistic model-agnostic meta-learning[C]//Proceedings of the 32nd Annual Conference?on Neural Information Processing Systems (NIPS’18), 2018: 9537-9548.
[68] COLLINS L, ?MOKHTARI A, ?SHAKKOTTAI S. Task- robust model?agnostic meta?learning[C]//Proceedings of the 34th Annual Conference?on Neural Information Processing Systems (NIPS’20), 2020: 18860-18871.
[69] 魏胜楠. 基于元学习的小样本图像分类方法研究[D]. 沈阳: 沈阳理工大学, 2023.
WEI S N. Research on few-shot image classification method based on meta-learning[D]. Shenyang: Shenyang Ligong University, 2023.
[70] 杜彦东, 冯林, 陶鹏, 等. 元迁移学习在少样本跨域图像分类中的研究[J]. 中国图象图形学报, 2023, 28(9): 2899-2912.
DU Y D, FENG L, TAO P, et al. Meta-transfer learning in cross-domain image classification with few-shot learning[J]. Journal of Image and Graphics, 2023, 28(9): 2899-2912.
[71] 李维刚, 甘平, 谢璐, 等. 基于样本对元学习的小样本图像分类方法[J]. 电子学报, 2022, 50(2): 295-304.
LI W G, GAN P, XIE L, et al. A few-shot image classification method by pairwise-based meta learning[J]. Acta Electronica Sinica, 2022, 50(2): 295-304.
[72] 江梦娟. 李群连续元学习算法研究[D]. 苏州: 苏州大学, 2022.
JIANG M J. Research on Lie group continual meta learning algorithm[D]. Suzhou: Soochow University, 2022.
[73] BATENI P, GOYAL R, MASRANI V. Improved few-shot visual classification[C]//Proceedings of the 33rd IEEE Conference on Computer Vision and Pattern Recognition(CVPR’20), 2020: 14481-14490.
[74] NGUYEN V N, LOKSE S, WICKSTROM K. SEN: a novel feature normalization dissimilarity measure for prototypical few?shot learning networks[C]//Proceedings of the 16th European Conference on Computer Vision (ECCV’20), 2020: 118-134.
[75] GIDARIS S, BURSUC A, KOMODAKIS N. Boosting few-shot visual learning with self-supervision[C]//Proceedings of the 32nd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19), 2019: 8059-8068.
[76] LI H, EIGEN D, DODGE S, et al. Finding task-relevant features for few-shot learning by category traversal[C]//Proceedings of the 32nd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19), 2019: 1-10.
[77] WANG X, YU F, WANG R, et al. TAFE-Net: task-aware feature embeddings for low shot learning[C]//Proceedings of the 32nd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19), 2019: 1831-1840.
[78] YE H J, HU H X, ZHAN D C. Learning embedding adaptation for few-shot learning with set-to-set functions[C]//Proceedings of the 33rd IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020: 8805-8814.
[79] HUANG W, YUAN Z, YANG A, et al. TAE-Net: task-adaptive embedding network for few-shot remote sensing scene classification[J]. Remote Sensing, 2022, 14: 111-130.
[80] WU F, SMITH J S, LU W. Attentive prototype few-shot learning with capsule network-based embedding[C]//Proceedings of the 16th European Conference on Computer Vision (ECCV’20), 2020: 237-253.
[81] LIU Y, LEE J, PARK M. Learning to propagate labels: transductive propagation network for few-shot learning[C]//Proceedings of the 7th International Conference on Learning Representations (ICLR’19), 2019: 1-8.
[82] SAHOO D, ?HUNG L H, ?LIU C H, ?et al. Meta domain adaptation: meta-learning for few-shot learning under domain?shift[C]//Proceedings of the 7th International Conference on Learning Representations (ICLR’19), 2019: 1-8.
[83] ZHAO A, DING M Y, LU Z W, et al. Domain-adaptive few-shot learning[J]. arXiv:2003.08626v1, 2020.
[84] TSENG H Y, LEE H Y, HUANG J B, et al. Cross-domain few-shot classification via learned feature-wise transformation[C]//Proceedings of the 8th International Conference on Learning Representations (ICLR’20), 2020: 1-8.
[85] GUAN J C, LU Z W, XIANG T, et al. Few-shot learning as domain adaptation: algorithm and analysis[C]//Proceedings of the 37th International Conference on Machine Learning(ICML’20), 2020: 1-10.
[86] KIM D, KIM J, CHO S, et al. Universal few-shot learning of dense prediction tasks with visual token matching[C]//Proceedings of the 11th International Conference on Learning Representations (ICLR’23), 2023: 1-26.
[87] LAI J X, YANG S Q, ZHOU J H, et al. Clustered-patch element connection for few-shot learning[C]//Proceedings of the 32nd International Joint Conference on Artificial Intelligence (IJCAI’23), 2023: 991-998.
[88] YANG Z Y, WANG J H, ZHU Y Y. Few-shot classification with contrastive learning[C]//Proceedings of the 18th European Conference on Computer Vision (ECCV’22), 2022: 293-309.