Academic Resource Text Hierarchical Multi-Label Classification

doi:10.3778/j.issn.1002-8331.2203-0286

Abstract

Abstract: The hierarchical multi-label text classification of scientific resource is used to assign scientific resource texts to a label system with a hierarchical structure. A text-level multi-label classification algorithm for scientific resources based on attention mechanism is proposed. The attention mechanism layer is constructed by integrating features such as text, keywords, and hierarchy message, which can be used to improve HMCN-F network to classify scientific and technological resource documents into the most relevant categories. In detail, word2vec and BiLSTM are mainly used to obtain the embedded vector and latent vector representation of text, keywords and hierarchical structures; the hierarchical attention mechanism is used to capture the correlation between keywords, label hierarchy and text word vectors to strengthen key words. The weight of the vector is used to generate a hierarchy-specific document embedding vector, which replaces the original text embedding in HMCN-F. The experimental results verify the effectiveness of the AHMCA method.

Key words: hierarchical multi-label classification, attention mechanism, BiLSTM, word2vec

摘要： 科技资源文本层次多标签分类（hierarchical multi-label text classification，HMTC）用于将科技资源文本分配到一个具有层级结构的标签体系中。提出基于注意力机制的科技资源文本层次多标签分类算法（academic resource text hierarchical multi-label classification based on attention，AHMCA）。通过整合文本、关键词、层次结构等特征构造注意力机制层，对HMCN-F（hierarchical multi-label classification network-feed-forward）网络进行改进，将科技资源文档逐级分类到最相关的类别中。细节上，主要利用word2vec与BiLSTM来获得文本、关键词、层次结构的嵌入向量和隐向量表示；利用层次注意力机制捕获关键词、标签层次结构与文本词向量之间的关联关系来强化重点词向量的权重，从而生成特定于层级的文档嵌入向量，替代HMCN-F中原始的文本嵌入。实验结果验证了AHMCA方法的有效性。

关键词: 层次多标签分类, 注意力机制, BiLSTM, word2vec

WANG Yue, LI Yawen, LI Ang. Academic Resource Text Hierarchical Multi-Label Classification[J]. Computer Engineering and Applications, 2023, 59(13): 92-98.

王岳, 李雅文, 李昂. 科技资源文本层次多标签分类方法[J]. 计算机工程与应用, 2023, 59(13): 92-98.

References

[1] WEHRMANN J，CERRI R，BARROS R.Hierarchical multi-label classification networks[C]//Proceedings of International Conference on Machine Learning，2018：5225-5234.
[2] FALL C J，T?RCSVáRI A，BENZINEB K，et al.Automated categorization in the international patent classification[C]//Proceedings of ACM Sigir Forum，2003：10-25.
[3] CESA-BIANCHI N，GENTILE C，ZANIBONI L.Incremental algorithms for hierarchical classification[J].Journal of Machine Learning Research，2006，7（1）：31-54.
[4] ROUSU J，SAUNDERS C，SZEDMAK S，et al.Kernel-based learning of hierarchical multilabel classification models[J].Journal of Machine Learning Research，2006，7（1）：1601-1626.
[5] MENG D Y，JIA Y M，DU J P，et al.Data-driven control for relative degree systems via iterative learning[J].IEEE Transactions on Neural Networks，2011，22（12）：2213-2225.
[6] CUI W Q，DU J P，WANG D W，et al.MVGAN：multi-view graph attention network for social event detection[J].ACM Transactions on Intelligent Systems and Technology，2021，12（3）：1-24.
[7] VENS C，STRUYF J，SCHIETGAT L，et al.Decision trees for hierarchical multi-label classification[J].Machine Learning，2008，73（2）：185.
[8] FANG Y K，DENG W H，DU J P，et al.Identity-aware CycleGAN for face photo-sketch synthesis and recognition[J].Pattern Recognition，2020，102：107249.
[9] CUI H Y，XUE Z，DU J P，et al.A novel topic extraction model for science and technology demand data[C]//Proceedings of the IEEE 7th International Conference on Cloud Computing and Intelligent Systems（CCIS），2021：444-448.
[10] CERRI R，BARROS R C，DE CARVALHO A C，et al.Reduction strategies for hierarchical multi-label classification in protein function prediction[J].BMC Bioinformatics，2016，17（1）：373.
[11] BORGES H B，NIEVOLA J C.Multi-label hierarchical classification using a competitive neural network for protein function prediction[C]//Proceedings of the 2012 International Joint Conference on Neural Networks（IJCNN），2012：1-8.
[12] SCHIETGAT L，VENS C，STRUYF J，et al.Predicting gene function using hierarchical multi-label decision tree ensembles[J].BMC Bioinformatics，2010，11（1）：1-14.
[13] KIRITCHENKO S，MATWIN S，FAMILI A F.Functional annotation of genes using hierarchical text categorization[C]//ACL Workshop on Linking Biological Literature，Ontologies and Databases：Mining Biological Semantics，2005.
[14] SCHAPIRE R E，SINGER Y.BoosTexter：a boosting-based system for text categorization[J].Machine Learning，2000，39（2）：135-168.
[15] KIM Y.Convolutional neural networks for sentence classification[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing，2014：1746-1751.
[16] PENG H，LI J，HE Y，et al.Large-scale hierarchical text classification with recursively regularized deep graph-CNN[C]//Proceedings of the World Wide Web Conference，2018：1063-1072.
[17] XUE Z，DU J P，DU D W，et al.Deep low-rank subspace ensemble for multi-view clustering[J].Information Sciences，2019，482：210-227.
[18] SILLA C N，FREITAS A A.A survey of hierarchical classification across different application domains[J].Data Mining and Knowledge Discovery，2011，22（1/2）：31-72.
[19] XUE Z，DU J P，ZHENG C W，et al.Clustering-induced adaptive structure enhancing network for incomplete multi-view data[C]//Proceedings of IJCAI 2021，2021：3235-3241.
[20] HU W M，GAO J，LI B，et al.Anomaly detection using local kernel density estimation and context-based regression[J].IEEE Transactions on Knowledge and Data Engineering，2018，32（2）：218-233.
[21] ZHANG L，XIAO K，LIU Q，et al.Modeling social attention for stock analysis：an influence propagation perspective[C]//Proceedings of 2015 IEEE International Conference on Data Mining，2015：609-618.
[22] LI W L，JIA Y M，DU J P.Variance-constrained state estimation for nonlinearly coupled complex networks[J].IEEE Transactions on Cybernetics，2017，48（2）：818-824.
[23] MENG D Y，JIA Y M，DU J P，et al.Initial shift problem for robust iterative learning control systems with polytopic?type uncertainty[J].International Journal of Systems Science，2010，41（7）：825-838.
[24] HUANG Z，LIU Q，CHEN E，et al.Question difficulty prediction for reading problems in standard tests[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence，2017.
[25] TAO H，TONG S，ZHAO H，et al.A radical-aware attention-based model for Chinese text classification[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence，2019.
[26] LIN Z，FENG M，SANTOS C N D，et al.A structured self-attentive sentence embedding[J].arXiv：1703.03130，2017.
[27] LIANG Z Y，DU J P，LI C Y.Abstractive social media text summarization using selective reinforced Seq2Seq attention model[J].Neurocomputing，2020，410：432-440.
[28] LIU P，QIU X，HUANG X.Recurrent neural network for text classification with multi-task learning[J].arXiv：1605.05101，2016.
[29] YIN W，SCHüTZE H.Attentive convolution：equipping CNNs with RNN?style attention mechanisms[J].Transactions of the Association for Computational Linguistics，2018，6：687-702.
[30] JOHNSON R，ZHANG T.Deep pyramid convolutional neural networks for text categorization[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2017：562-570.
[31] SHIMURA K，LI J，FUKUMOTO F.HFT-CNN：learning hierarchical category structure for multi-label short text categorization[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing，2018：811-816.