属性蒸馏的零样本识别方法

doi:10.3778/j.issn.1002-8331.2212-0382

摘要/Abstract

摘要： 零样本识别是计算机视觉领域最具挑战性的任务之一，其关键在于如何从已见类中学到稳定和可迁移的知识。为提高零样本识别的准确率，对零样本识别问题进行了系统研究，并利用知识蒸馏的思想，精心设计了一个简单有效的属性蒸馏分类器。它符合人类认识事物的过程，首先从Vision Transformer大模型中获得全面细致的视觉特征，再运用属性概念蒸馏出物体的属性知识，最后迁移到未见类识别任务中。公开数据集上的实验表明，该方法取得了具有竞争力的结果，其识别准确率虽略低于最新的属性引导算法，但优于其他传统方法，而且识别架构简单具有更快的处理速度。同时，研究也指出了减少属性描述的稀疏性，以及增加多视角高清图像，将有利于提高零样本识别方法的准确率。

关键词: 计算机视觉, 零样本识别, 知识蒸馏, 属性蒸馏

Abstract: Zero-shot recognition is one of the most challenging tasks in the field of computer vision. The key problem is how to learn stable and transferable knowledge from the seen class. In order to increase the accuracy of zero-shot recognition, this paper carefully investigates the issue of zero-shot recognition and develops a straight forward and efficient attribute-distillation classifier based on the notion of knowledge distillation. It is consistent with how people generally understand things. It begins by obtaining extensive and precise visual features from the large model Vision Transformer, then uses the attribute idea to extract the attribute knowledge of objects before transforming to the task of classifying unseen classes. Public dataset experiments demonstrate that the proposed method has produced results that are competitive. Its recognition accuracy is slightly below that of the most recent attribute-guided algorithm, but it is still better than other conventional approaches, and its simple recognition architecture can achieve fast processing speed. Nevertheless, this research also makes the point that decreasing the sparsity of attribute descriptions and increasing multi-view high-definition photos will contribute to an increase in zero-shot recognition accuracy.

Key words: computer vision, zero-shot recognition, knowledge distillation, attribute distillation

李厚君, 韦柏全. 属性蒸馏的零样本识别方法[J]. 计算机工程与应用, 2024, 60(9): 219-227.

LI Houjun, WEI Boquan. Attribute Distillation for Zero-Shot Recognition[J]. Computer Engineering and Applications, 2024, 60(9): 219-227.

参考文献

[1] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409.1556, 2014.
[2] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[3] SZEGEDY C, IOFFE S, VANHOUCKE V, et al. Inception-v4, inception-resnet and the impact of residual connections on learning[C]//Thirty-First AAAI Conference on Artificial Intelligence, 2017.
[4] ZHU X X , ANGUELOV D, RAMANAN D. Capturing long-tail distributions of object subcategories[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014: 915-922.
[5] ROHRBACH M, STARK M, SCHIELE B. Evaluating knowledge transfer and zero-shot learning in a large-scale setting[C]//2011 IEEE Conference on Computer Vision and Pattern Recognition, 2011: 1641-1648.
[6] AKATA Z, PERRONNIN F, HARCHAOUI Z, et al. Label embedding for attribute-based classification[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013: 819-826.
[7] AKATA Z, REED S, WALTER D, et al. Evaluation of output embeddings for fine-grained image classification[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015: 2927-2936.
[8] LAROCHELLE H, ERHAN D, BENGIO Y. Zero-data learning of new tasks[C]//Proceedings of the 23rd National Conference on Artificial Intelligence, 2008: 646-651.
[9] PALATUCCI M, POMERLEAU D, HINTON G E, et al. Zero-shot learning with semantic output codes[C]//Advances in Neural Information Processing Systems, 2009: 1410-1418.
[10] LAMPERT C H, NICKISCH H, HARMELING S. Learning to detect unseen object classes by between-class attribute transfer[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009: 951-958.
[11] LAMPERT C H, NICKISCH H, HARMELING S. Attribute-based classification for zero-shot visual object categorization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 36(3): 453-465.
[12] AKATA Z, PERRONNIN F, HARCHAOUI Z, et al. Label-embedding for image classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(7): 1425-1438.
[13] XIAN Y Q, AKATA Z SHARMA G, et al. Latent embeddings for zero-shot classification[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 69-77.
[14] ROMERA-PAREDES B, TORR P. An embarrassingly simple approach to zero-shot learning[C]//International Conference on Machine Learning, 2015: 2152-2161.
[15] VERMA V K, RAI P. A simple exponential family framework for zero-shot learning[C]//Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Cham: Springer, 2017: 792-808.
[16] ZHANG L, WANG P, LIU L Q, et al. Towards effective deep embedding for zero-shot learning[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(9): 2843-2852.
[17] XIE G S, LIU L, JIN X B, et al. Attentive region embedding network for zero-shot learning[C]//2019 IEEE Conference on Computer Vision and Pattern Recognition, 2019: 9376-9385.
[18] XIE G S, LIU L, ZHU F, et al. Region graph embedding network for zero-shot learning[C]//Proceedings of 16th European Conference on Computer Vision, Glasgow, UK, August 23-28, 2020. [S.l.]: Springer International Publishing, 2020: 562-580.
[19] ZHU Y Z, XIE J W, TANG Z Q, et al. Semantic-guided multi-attention localization for zero-shot learning[C]//2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing-NeurIPS Edition (EMC2-NIPS 2019), Held, 13 December 2019. Vancouver, British Columbia, Canada: IEEE, 2019.
[20] LIU Y, ZHOU L, BAI X, et al. Goal-oriented gaze estimation for zero-shot learning[C]//2021 IEEE Conference on Computer Vision and Pattern Recognition, 2021: 3794-3803.
[21] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//2017 Advances in Neural Information Processing Systems, 2017: 5998-6008.
[22] CHEN S M, HONG Z M, LIU Y, et al. TransZero: attribute-guided transformer for zero-shot learning[C]//2022 Proceedings of the AAAI Conference on Artificial Intelligence, 2022: 330-338.
[23] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[C]//2021 International Conference on Learning Representations, 2021.
[24] ZHU X Z, SU W J, LU L W, et al. Deformable detr: deformable transformers for end-to-end object detection[C]//2021 International Conference on Learning Representations, 2021.
[25] TOUVRON H, CORD M, DOUZE M, et al. Training data-efficient image transformers & distillation through attention[C]//International Conference on Machine Learning, 2021: 10347-10357.
[26] LI Y J, SHAN C H, LI H J, et al. A capsule-unified framework of deep neural networks for graphical programming[J]. Soft Computing, 2021, 25: 3849-3871.
[27] WAH C, BRANSON S, WELINDER P, et al. Caltech-UCSD birds 200, CNS-TR-2010-001[R]. California Institute of Technology, 2010.
[28] XIAN Y Q, LAMPERT C H, SCHIELE B, et al. Zero-shot learning-a comprehensive evaluation of the good, the bad and the ugly[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(9): 2251-2265.