基于特征分布学习的小样本类增量学习

doi:10.3778/j.issn.1002-8331.2205-0385

计算机工程与应用 ›› 2023, Vol. 59 ›› Issue (14): 151-157.DOI: 10.3778/j.issn.1002-8331.2205-0385

基于特征分布学习的小样本类增量学习

姚光乐，祝钧桃，周文龙，张贵宇，张伟，张谦

1.人工智能四川省重点实验室，四川宜宾 643000
2.成都理工大学计算机与网络安全学院，成都 610059
3.四川轻化工大学自动化与信息工程学院，四川宜宾 643000
4.电子科技大学信息与通信工程学院，成都 611731
5.电子信息控制重点实验室，成都 610036

出版日期:2023-07-15 发布日期:2023-07-15

Few-Shot Class-Incremental Learning Based on Feature Distribution Learning

YAO Guangle, ZHU Juntao, ZHOU Wenlong, ZHANG Guiyu, ZHANG Wei, ZHANG Qian

1.Artificial Intelligence Key Laboratory of Sichuan Province, Yibin, Sichuan 643000, China
2.School of Computer and Network Security, Chengdu University of Technology, Chengdu 610059, China
3.School of Automation & Information Engineering, Sichuan University of Science & Engineering, Yibin, Sichuan 643000, China
4.School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
5.Science and Technology on Electronic Information Control Laboratory, Chengdu 610036, China

Online:2023-07-15 Published:2023-07-15

摘要/Abstract

摘要： 关注了一个非常具有挑战性的问题：深度神经网络的小样本类增量学习。其中深度神经网络模型可以从少量的样本中逐步学习新知识，同时不会忘记已学习的旧知识。为了平衡模型对旧知识的记忆和对新知识的学习，提出了一个基于特征分布学习的小样本类增量学习方法。在基类上学习模型以获得一个性能良好的特征提取器，并使用每类的特征分布信息来表示知识。将已学习的知识与新类的特征一起映射到一个新的低维子空间中，以统一地回顾旧知识与学习新知识。在子空间内，还为每个新类生成了分类权值初始化，以提高模型对新类的适应性。大量实验表明，该方法可以有效地减轻模型对已学习知识的遗忘，同时提高模型对新知识的适应性。

关键词: 小样本类增量学习, 深度神经网络, 增量学习

Abstract: This paper focuses on a very challenging problem：few-shot class-incremental learning for deep neural networks, where the deep neural network model can gradually learn new knowledge from a small number of samples without forgetting the learned knowledge. To balance the model’s memory of old knowledge and learning of new knowledge, it proposes a few-shot class-incremental learning method based on feature distribution learning. First, it learns the model on the base classes to obtain a well-performing feature extractor and take the feature distribution information as the learned knowledge. Then, it maps the learned knowledge together with the features of the novel classes into a low-dimensional subspace to review old knowledge and learn new knowledge uniformly. Finally, within the subspace, it also generates classification weight initializations for each novel class to improve the adaptability of the model to novel classes. Extensive experiments show that the method can effectively alleviate the model’s forgetting of learned knowledge and improve the model’s adaptability to new knowledge.

Key words: few-shot class-incremental learning, deep neural network, incremental learning

姚光乐, 祝钧桃, 周文龙, 张贵宇, 张伟, 张谦. 基于特征分布学习的小样本类增量学习[J]. 计算机工程与应用, 2023, 59(14): 151-157.

YAO Guangle, ZHU Juntao, ZHOU Wenlong, ZHANG Guiyu, ZHANG Wei, ZHANG Qian. Few-Shot Class-Incremental Learning Based on Feature Distribution Learning[J]. Computer Engineering and Applications, 2023, 59(14): 151-157.

参考文献

[1] TAO X，HONG X，CHANG X，et al.Few-shot class-incremental learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2020：12180-12189.
[2] DOUILLARD A，CORD M，OLLION C，et al.PODNet：pooled outputs distillation for small-tasks incremental learning[C]//European Conference on Computer Vision，2020.
[3] GIDARIS S，KOMODAKIS N.Generating classification weights with GNN denoising autoencoders for few-shot learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：21-30.
[4] TAO X，HONG X，SHI W，et al.Analogy-detail networks for object recognition[J].IEEE Transactions on Neural Networks and Learning Systems，2021，32：4404-4418.
[5] RAJASEGARAN J，KHAN S，HAYAT M，et al.iTAML：an incremental task-agnostic meta-learning approach[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2020：13585-13594.
[6] YU L，TWARDOWSKI B，LIU X，et al.Semantic drift compensation for class-incremental learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2020：6980-6989.
[7] CHERAGHIAN A，RAHMAN S，FANG P.Semantic-aware knowledge distillation for few-shot class-incremental learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2021：2534-2543.
[8] 韩纪东，李玉鑑.神经网络模型中灾难性遗忘研究的综述[J].北京工业大学学报，2021，47（5）：551-564.
HAN J D，LI Y J.Survey of catastrophic forgetting research in neural network models[J].Journal of Beijing University of Technology.2021，47（5）：551-564.
[9] REBUFFI S A，KOLESNIKOV A，SPERL G，et al.iCaRL：incremental classifier and representation learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：5533-5542.
[10] HOU S，PAN X，LOY C C，et al.Learning a unified classifier incrementally via rebalancing[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：831-839.
[11] KRIZHEVSKY A，HINTON G.Learning multiple layers of features from tiny images[J].Handbook of Systemic Autoimmune Diseases，2009，1（4）.
[12] VINYALS O，BLUNDELL C，LILLICRAP T，et al.Matching networks for one shot learning[C]//Advances in Neural Information Processing Systems，2016：3630-3638.
[13] LECUN Y，BENGIO Y，HINTON G.Deep learning[J].Nature，2015，521（7553）：436-444.
[14] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[15] RUSSAKOVSKY O，DENG J，SU H，et al.Imagenet large scale visual recognition challenge[J].International Journal of Computer Vision，2015，115（3）：211-252.
[16] CASTRO F M，MAR′IN-JIM′ENEZ M J，GUIL N，et al.End-to-end incremental learning[C]//European Conference on Computer Vision，2018.
[17] 莫建文，陈瑶嘉.基于VAE的伪样本重排练实现的类增量学习[J].计算机工程与设计，2021，42（8）：2284-2290.
MO J W，CHEN Y J.Class incremental learning based on pseudo-rehearsal with variational autoencoder[J].Computer Engineering and Design，2021，42（8）：2284-2290.
[18] LI Z，HOIEM D.Learning without forgetting[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2018，40（12）：2935-2947.
[19] KIRKPATRICK J，PASCANU R，RABINOWITZ N，et al.Overcoming catastrophic forgetting in neural networks[J].Proceedings of the National Academy of Sciences of the United States of America，2017，114：3521-3526.
[20] REN M，LIAO R，FETAYA E，et al.Incremental few-shot learning with attention attractor networks[C]//Advances in Neural Information Processing Systems，2019.
[21] GIDARIS S，KOMODAKIS N.Dynamic few-shot visual learning without forgetting[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4367-4375.

基于特征分布学习的小样本类增量学习

Few-Shot Class-Incremental Learning Based on Feature Distribution Learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	崔少国, 独潇, 杨泽田. 多注意力机制融合低高阶特征的神经推荐算法[J]. 计算机工程与应用, 2023, 59(8): 192-199.
[2]	梁义涛, 韩永波, 李磊. 深度长时目标跟踪算法综述[J]. 计算机工程与应用, 2023, 59(4): 1-17.
[3]	周坤, 徐黎明, 郑伯川, 谢亦才. 自适应高效深度跨模态增量哈希检索算法[J]. 计算机工程与应用, 2023, 59(2): 85-93.
[4]	王少华, 刘法胜, 时柏营, 刘兴波, 聂秀山. 面向交通标识的二值语义嵌入学习方法[J]. 计算机工程与应用, 2023, 59(13): 205-210.
[5]	刘金琳, 李冬冬, 王喆, 蔡立志. 两级特征联合学习的情感说话人识别[J]. 计算机工程与应用, 2023, 59(1): 149-155.
[6]	杨曦, 闫杰, 王文, 李少毅, 林健. 脑启发的视觉目标识别模型研究与展望[J]. 计算机工程与应用, 2022, 58(7): 1-20.
[7]	马幪浩, 王喆. 小样本下基于Wasserstein距离的半监督学习算法[J]. 计算机工程与应用, 2022, 58(5): 193-199.
[8]	徐岩柏, 景运革. 多源数据矩阵增量约简算法[J]. 计算机工程与应用, 2022, 58(3): 195-200.
[9]	贠璟扬, 李学华, 向维. 语义导向多尺度多视图深度估计算法[J]. 计算机工程与应用, 2022, 58(2): 215-224.
[10]	王瑞平, 吴士泓, 张美航, 王小平. 视觉问答语言处理方法综述[J]. 计算机工程与应用, 2022, 58(17): 50-60.
[11]	于强, 林民, 李艳玲. 基于深度学习的关键词生成研究综述[J]. 计算机工程与应用, 2022, 58(14): 27-39.
[12]	王欣然, 田启川, 张东. 人脸口罩佩戴检测研究综述[J]. 计算机工程与应用, 2022, 58(10): 13-26.
[13]	许昊，张凯，田英杰，种法广，王子超. 深度神经网络图像描述综述[J]. 计算机工程与应用, 2021, 57(9): 9-22.
[14]	王林，柴江云. 深度神经网络在多场景车辆属性识别中的研究[J]. 计算机工程与应用, 2021, 57(9): 162-167.
[15]	祝钧桃，姚光乐，张葛祥，李军，杨强，王胜，叶绍泽. 深度神经网络的小样本学习综述[J]. 计算机工程与应用, 2021, 57(7): 22-33.