Multi-Feature Relationship Classification Algorithm Fused with Fine-Grained Entity Types

doi:10.3778/j.issn.1002-8331.2106-0278

Abstract

Abstract: In recent years, relationship classification based on deep learning has mostly focused on the improvement of attention mechanism or the optimization of semantic information, but the features extracted by such methods are often relatively single, ignoring the impact of entity types on relationship classification, and there is semantic information problem such as incomplete learning. This paper proposes a new relationship classification method Type-SBNE, which introduces fine-grained entity type information for entity type learning tasks, and generates entity class vectors by averaging all entity vectors in each entity type, and then the corresponding feature vectors are obtained through entity information learning and sentence meaning information learning, and composite semantic features are obtained through splicing and fusion. Finally, the fully connected layer and Softmax function are used to predict the relationship between entity pairs. Based on the fine-grained entity type information, the characteristics of the entity are enriched, and the expression of each entity in the context is effectively strengthened. Experiments show that the Type-SBNE model can better complete the relationship classification task, and the effect is improved compared with other algorithms.

Key words: relation classification, BERT, Nystr?mformer, fine-grained entity type vector

摘要： 近年来，基于深度学习的关系分类多集中于注意力机制的改进或语义信息的优化两方面，但此类方法提取的特征往往较为单一，忽略了实体类型对关系分类的影响，且存在语义信息学习不完整等问题。提出一种新的关系分类方法Type-SBNE，针对实体类型学习任务，引入细粒度的实体类型信息，并通过对每个实体类型中的所有实体向量取平均生成实体类向量，再分别通过实体与句意信息学习获得其对应的特征向量，通过拼接融合得到复合语义特征，最后采用全连接层和Softmax函数来预测实体对之间的关系。Type-SBN基于细粒度的实体类型信息，丰富了实体的特征，有效加强了每个实体在上下文中的表达。实验表明，Type-SBNE模型可更好地完成关系分类任务，效果优于对比方法。

关键词: 关系分类, BERT, Nystr?mformer, 细粒度实体类向量

ZUO Yayao, YI Biao, LI Wenjie. Multi-Feature Relationship Classification Algorithm Fused with Fine-Grained Entity Types[J]. Computer Engineering and Applications, 2022, 58(22): 65-71.

左亚尧, 易彪, 黎文杰. 融合细粒度实体类型的多特征关系分类算法[J]. 计算机工程与应用, 2022, 58(22): 65-71.

References

[1] CHEN W，ZHA H，CHEN Z，et?al.HybridQA：a dataset of multi-hop question answering over tabular and textual data[C]//Findings of the Association for Computational Linguistics：EMNLP 2020，2020：1026-1036.
[2] 廖国琼，蓝天明，黄晓梅，等.基于事件社会网络推荐系统综述[J].软件学报，2021，32（2）：424-444.
LIAO G Q，LAN T M，HUANG X M，et al.Survey on recommendation systems in event-based social networks[J].Journal of Software，2021，32（2）：424-444.
[3] KAUR P，PANNU H S，MALHI A K.Comparative analysis on cross-modal information retrieval：a review[J].Computer Science Review，2021，39：100336.
[4] LAN Y，HAO Y，XIA K，et al.Stacked residual recurrent neural networks with cross-layer attention for text classification[J].IEEE Access，2020，8：70401-70410.
[5] HELMY A A，OMAR Y M K，HODHOD R.An innovative word encoding method for text classification using convolutional neural network[C]//2018 14th International Computer Engineering Conference，2018：42-47.
[6] HUAN H，YAN J，XIE Y，et al.Feature-enhanced nonequilibrium bidirectional long short-term memory model for Chinese text classification[J].IEEE Access，2020，8：199629-199637.
[7] 程威，邵一帆，钱龙华，等.基于最短依存路径和集成学习的化学物蛋白质关系抽取[J].中文信息学报，2021，35（4）：58-65.
CHENG W，SHAO Y F，QIAN L H，et al.Chemical protein relation extraction based on shortest dependency path and ensemble learning[J].Journal of Chinese Information Processing，2021，35（4）：58-65.
[8] 王兴.基于深度学习的语义关系分类研究[D].北京：北京交通大学，2018.
WANG X.Research on semantic relationship classification based on deep learning[D].Beijing：Beijing Jiaotong University，2018.
[9] 刘峰，高赛，于碧辉，等.基于Multi-head attention和Bi-LSTM的实体关系分类[J].计算机系统应用，2019，28（6）：118-124.
LIU F，GAO S，YU B H，et al.Relation classification based on multi-head attention and bidirectional long short-term memory networks[J].Computer Systems and Applications，2019，28（6）：118-124.
[10] HONG W，XU W，QI J，et?al.Neural tensor network for multi-label classification[J].IEEE Access，2019，7：96936-96941.
[11] ZHONG Y，XU M，WEI W，et?al.Multi-label classification method based on the labeling-importance degree[C]//2019 20th IEEE/ACIS International Conference on Software Engineering，Artificial Intelligence，Networking and Parallel/Distributed Computing，2019.
[12] ZHU Z，SU J，ZHOU Y.Improving distantly supervised relation classification with attention and semantic weight[J].IEEE Access，2019，7：91160-91168.
[13] WU S，HE Y.Enriching pre-trained language model with entity information for relation classification[C]//28th ACM International Conference on Information and Knowledge Management.New York：ACM，2019：2361-2364.
[14] 张东东，彭敦陆.ENT-BERT：结合BERT和实体信息的实体关系分类模型[J].小型微型计算机系统，2020，41（12）：2557-2562.
ZHANG D D，PENG D L.ENT-BERT：entity relation classification model combining BERT and entity information[J].Journal of Chinese Computer Systems，2020，41（12）：2557-2562.
[15] 万莹，孙连英，赵平，等.基于信息增强BERT的关系分类[J].中文信息学报，2021，35（3）：69-77.
WAN Y，SUN L Y，ZHAO P，et al.Relation classification based on information enhanced BERT[J].Journal of Chinese Information Processing，2021，35（3）：69-77.
[16] HENDRICKX I，KIM S N，KOZAREVA Z，et?al.SemEval-2010 task 8：multi-way classification of semantic relations between pairs of nominals[C]//5th International Workshop on Semantic Evaluation，Uppsala，2010：33-38.
[17] RINK B，HARABAGIU S.UTD：classifying semantic relations by combining lexical and semantic resources[C]//5th International Workshop on Semantic Evaluation，Uppsala，2010：256-259.
[18] ZENG D，LIU K，LAI S，et?al.Relation classification via convolutional deep neural network[C]//25th International Conference on Computational Linguistics：Technical Papers，Dublin，2014：2335-2344.
[19] DOS SANTOS C，XIANG B，ZHOU B.Classifying relations by ranking with convolutional neural networks[C]//53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing （Volume 1：Long Papers）.Beijing，2015：626-634.
[20] SHEN Y，HUANG X.Attention-based convolutional neural network for semantic relation extraction[C]//26th International Conference on Computational Linguistics：Technical Papers，Osaka，2016：2526-2536.
[21] WANG L，CAO Z，DE?MELO G，et?al.Relation classification via multi-level attention CNNs[C]//54th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），Berlin，2016：1298-1307.
[22] GUO X，ZHANG H，YANG H，et?al.A single attention-based combination of CNN and RNN for relation classification[J].IEEE Access，2019，7：12467-12475.
[23] SOCHER R，HUVAL B，MANNING C D，et?al.Semantic compositionality through recursive matrix-vector spaces[C]//2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning，Jeju Island，2012：1201-1211.
[24] CUI Q，WU S，LIU Q，et?al.MV-RNN：a multi-view recurrent neural network for sequential recommendation[J].IEEE Transactions on Knowledge and Data Engineering，2020，32（2）：317-331.
[25] YU M，GORMLEY M R，DREDZE M.Factor-based compositional embedding models[C]//NIPS Workshop on Learning Semantics，2014：95-101.
[26] LEE J，SEO S，CHOI Y S.Semantic relation classification via bidirectional LSTM networks with entity-aware attention using latent entity typing[J].Symmetry，2019，11（6）：785.
[27] MENG B，BAOMIN X，ZHOU E，et?al.Bidirectional gated recurrent unit networks for relation classification with multiple attentions and semantic information[C]//International Symposium on Neural Networks.Cham：Springer，2019：124-132.