结合知识图谱的变分自编码器零样本图像识别

doi:10.3778/j.issn.1002-8331.2106-0430

摘要/Abstract

摘要： 近年来，结合生成模型的零样本算法得到了广泛的研究，但此类方法通常仅使用属性注释，缺少类别语义，而单一信息对类别表征能力不够强，容易产生域偏移，影响知识迁移的效果，进而降低分类结果的准确率。为了解决此问题，提出一种结合知识图谱变分自编码器零样本识别算法（KG-VAE），通过构建联合类别分级结构，类别文本描述和词向量的层次结构化知识图谱作为语义信息库，将知识图谱中丰富的语义知识结合到以变分自编码器为基础的生成模型中，使生成的潜在特征更好保留有效的判定性信息，减小域偏移，促进知识迁移。在四个公开的零样本数据集上进行了实验，对比基准方法CADA-VAE，分类平均准确率有一定的提高；同时利用消融实验证明了知识图谱作为语义辅助信息的有效性。

关键词: 知识图谱, 图卷积神经网络, 图变分自编码器, 零样本学习, 变分自编码器

Abstract: Recently, zero-shot learning combine with generative model has been widely studied, but such methods usually only use attributes, lacking class semantics, a single information cannot strong enough to represent the class. It is easy to cause the domain-shift problem and affect knowledge transfer, the accuracy of classification results will be decreased. In order to solve this problem, variational auto-encoder combined with knowledge graph zero-shot learning（KG-VAE） is proposed. By building hierarchical structured knowledge graph with class description and word vector as the semantic information, the rich semantic information in the knowledge graph is combined into the VAE model. The generated potential features retain better effective deterministic information, it is effective to reduce domain shift and promote knowledge transfer. According to the evaluation with four public datasets, and compared with the baseline method CADA-VAE, the average classification accuracy is improved. At the same time, the availability of knowledge graph is proved by ablation experiment.

Key words: knowledge graph, graph convolutional network, variational graph auto-encoder, zero-shot learning, variational auto-encoder

张海涛, 苏琳. 结合知识图谱的变分自编码器零样本图像识别[J]. 计算机工程与应用, 2023, 59(1): 236-243.

ZHANG Haitao, SU Lin. Variational Auto-Encoder Combined with Knowledge Graph Zero-Shot Learning[J]. Computer Engineering and Applications, 2023, 59(1): 236-243.

参考文献

[1] FROME A，CORRADO G，SHLENS J，et al.Devise：A deep visual-semantic embedding model[C]//Proceedings of Advances in Neural Information Processing Systems，2013：2121-2129.
[2] AKATA Z，PERRONNIN F，HARCHAOUI Z，et al.Label-embedding for image classification[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，38（7）：1425-1438.
[3] AKATA Z，REED S，WALTER D，et al.Evaluation of output embeddings for fine-grained image classification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：2927-2936
[4] ROMERA-PAREDES B，TORR P.An embarrassingly simple approach to zero-shot learning[C]//Proceedings of the International Conference on Machine Learning，2015：2152-2161.
[5] XIAN Y，AKATA Z，SHARMA G，et al.Latent embeddings for zero-shot classification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：69-77.
[6] CHANGPINYO S，CHAO W L，GONG B，et al.Synthesized classifiers for zero-shot learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：5327-5336.
[7] KODIROV E，XIANG T，GONG S.Semantic autoencoder for zero-shot learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3174-3183.
[8] GOODFELLOW I，POUGET-ABADIE J，MIRZA M，et al.Generative adversarial nets[C]//Proceedings of Advances in Neural Information Processing Systems，2014：2672-2680.
[9] KINGMA D P，WELLING M.Auto-encoding variational Bayes[J].arXiv：1312.6114，2013.
[10] XIAN Y，LORENZ T，SCHIELE B，et al.Feature generating networks for zero-shot learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：5542-5551.
[11] MISHRA A，KRISHNA REDDY S，MITTAL A，et al.A generative model for zero shot learning using conditional variational autoencoders[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2018：2188-2196.
[12] SCHONFELD E，EBRAHIMI S，SINHA S，et al.Generalized zero-and few-shot learning via aligned variational autoencoders[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：8247-8255.
[13] VERMA V K，ARORA G，MISHRA A，et al.Generalized zero-shot learning via synthesized examples[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4281-4289.
[14] ZHU Y，XIE J，LIU B，et al.Learning feature-to-feature translator by alternating back-propagation for generative zero-shot learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：9844-9854.
[15] FU Y，HOSPEDALES T M，XIANG T，et al.Transductive multi-view zero-shot learning[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（11）：2332-2345.
[16] LAMPERT C H，NICKISCH H，HARMELING S.Attribute-based classification for zero-shot visual object categorization[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2013，36（3）：453-465.
[17] SHIGETO Y，SUZUKI I，HARA K，et al.Ridge regression，hubness，and zero-shot learning[C]//Joint European conference on Machine Learning and Knowledge Discovery in Databases，2015：135-151.
[18] GULRAJANI I，AHMED F，ARJOVSKY M，et al.Improved training of wasserstein gans[J].arXiv：1704.00028，2017.
[19] WANG X，YE Y，GUPTA A.Zero-shot recognition via semantic embeddings and knowledge graphs[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：6857-6866.
[20] KAMPFFMEYER M，CHEN Y，LIANG X，et al.Rethinking knowledge graph propagation for zero-shot learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：11487-11496.
[21] LIU L，ZHOU T，LONG G，et al.Attribute propagation network for graph zero-shot learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：4868-4875.
[22] KIPF T N，WELLING M.Semi-supervised classification with graph convolutional networks[J].arXiv：1609.02907，2016.
[23] HIGGINS I，MATTHEY L，PAL A，et al.Beta?VAE：Learning basic visual concepts with a constrained variational framework[C]//Proceedings of ICLR 2017 Conference，2017.
[24] CHAO W L，CHANGPINYO S，GONG B，et al.An empirical study and analysis of generalized zero-shot learning for object recognition in the wild[C]//Proceedings of the European Conference on Computer Vision，2016：52-68.
[25] KIPF T N，WELLING M.Variational graph auto-encoders[J].arXiv：1611.07308，2016.
[26] KINGMA D P，BA J.ADAM：A method for stochastic optimization[J].arXiv：1412.6980，2014.
[27] XIAN Y，SCHIELE B，AKATA Z.Zero-shot learning-the good，the bad and the ugly[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：4582-4591.