Generalization Performance Optimization of Entity Link Models Based on Multi-Channel Feature Fusion

doi:10.3778/j.issn.1002-8331.2205-0383

Abstract

Abstract: Entity linking is a key link in knowledge base question answering and knowledge graph construction. The semantic expression of the Chinese corpus is sparse, and there are a large number of similar entities that are difficult to distinguish. The general model is too dependent on the feature information other than the original question and answer. It is difficult to fully learn text features, which makes it difficult to improve the accuracy of entity linking, in turn limits the performance upper limit of upper-level applications such as question answering. Aiming at these problems, this paper focuses on the candidate generation and candidate disambiguation of entity links in question answering systems, and regards entity disambiguation as a classification task. A multi-channel network model based on Bi-LSTM and CNN is constructed, and a threshold weight splicing strategy is proposed to fuse the multidimensional features extracted by CNN and LSTM channels. The bidirectional attention mechanism is introduced to fully mine the deep semantic relationship between the question mention representation and the knowledge base entity description, and effectively reduce the dependence of question answering on additional feature rules, so that it can be applied in multi-domain knowledge bases. The experimental results show that the proposed entity linking model significantly improves the overall performance of the question answering system, and has strong generalization under the condition of only relying on the original information of question answering. The optimal [Acc@1] and [F1] values ??are obtained on CCKS2019-CKBQA and NLPCC-2016KBQA datasets.

Key words: knowledge base question answering（KBQA）, entity link, multi-channel, feature fusion, bidirectional attention mechanism, generalization performance

摘要： 实体链接是知识库问答和知识图谱构建的关键环节，中文语料库的语义表达稀疏，存在大量难以区分的相似实体，一般模型过于依赖除原始问答以外的特征信息，很难完全学习文本特征，使得实体链接准确率难以提高，进而限制了问答等上层应用的性能上限。针对这些问题，聚焦问答系统实体链接的候选生成和候选消歧，将实体消歧视为分类任务，构建了一种基于Bi-LSTM和CNN的多通道网络模型，提出阈值权重拼接策略融合CNN和LSTM通道提取的多维特征。引入双向注意力机制，充分挖掘问句提及表征和知识库实体描述之间的深层语义关系，有效降低问答对额外特征规则的依赖，以便应用在多领域知识库中。实验结果表明，在仅依靠问答原始信息的情况下，提出的实体链接模型显著提高了问答系统的整体性能，并具有较强的泛化性，在公开数据集CCKS2019-CKBQA和NLPCC-2016KBQA中取得了最优的[Acc@1]和[F1]值。

关键词: 知识库问答, 实体链接, 多通道, 特征融合, 双向注意力机制, 泛化性能

CHEN Yang, WAN Weibing. Generalization Performance Optimization of Entity Link Models Based on Multi-Channel Feature Fusion[J]. Computer Engineering and Applications, 2023, 59(16): 125-134.

陈阳, 万卫兵. 多通道特征融合的实体链接模型泛化性能优化[J]. 计算机工程与应用, 2023, 59(16): 125-134.

References

[1] BIZER C，LEHMANN J，KOBILAROV G.DBpedia—a crystallization point for the web of data[J].Journal of Web Semantics，2009，7（3）：154-165.
[2] MAHDISOLTANI F，BIEGA J，SUCHANEK F.YAGO3：a knowledge base from multilingual Wikipedias[C]//Proceedings of the 7th Biennial Conference on Innovative Data Systems Research，2014.
[3] BOLLACKER K，EVANS C，PARITOSH P.FreeBase：a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data，2008.
[4] SOWA J F.Principles of semantic networks：explorations in the representation of knowledge[M].San Fransisco：Morgan Kaufmann，2014.
[5] SHEN W，WANG J，HAN J.Entity linking with a knowledge base：issues，techniques，and solutions[J].IEEE Transactions on Knowledge and Data Engineering，2014，27（2）：443-460.
[6] BERANT J，CHOU A，FROSTIG R.Semantic parsing on freebase from question-answer pairs[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing，2013：1533-1544.
[7] ZETTLEMOYER L S，COLLINS M.Learning to map sentences to logical form：structured classification with probabilistic categorial grammars[J].arXiv：1207.1420.2012.
[8] KWIATKOWSKI T，ZETTLEMOYER L，GOLDWATER S.Lexical generalization in CCG grammar induction for semantic parsing[C]//Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing，2011：1512-1523.
[9] REDDY S，LAPATA M，STEEDMAN M.Large-scale semantic parsing without question-answer pairs[J].Transactions of the Association for Computational Linguistics，2014，2：377-392.
[10] YIH S W，CHANG M，HE X.Semantic parsing via staged query graph generation：question answering with knowledge base[C]//Proceedings of the the 53rd Annual Meeting of the ACL and the 7th International Joint Conference on Natural Language Processing of the AFNLP，2015.
[11] STERN R，SAGOT B，BéCHET F.A joint named entity recognition and entity linking system[C]//Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data，2012：52-60.
[12] YU M，YIN W，HASAN K S.Improved neural relation detection for knowledge base question answering[J].arXiv：1704.06194.2017.
[13] BORDES A，USUNIER N，CHOPRA S，et al.Large-scale simple question answering with memory networks[J].arXiv：1506.02075.2015.
[14] TALMOR A，BERANT J.The web as a knowledge-base for answering complex questions[J].arXiv：1803.06643，2018.
[15] USBECK R，GUSMITA R H，MUHAMMAD M，et al.9th challenge on question answering over linked data（QALD-9）[J].Language，2018，7（1）：58-64.
[16] ZHANG L，LIN C，ZHOU D.A Bayesian end-to-end model with estimated uncertainties for simple question answering over knowledge bases[J].Computer Speech & Language，2021，66：101167.
[17] HAO Y，LIU H，HE S.Pattern-revising enhanced simple question answering over knowledge bases[C]//Proceedings of the 27th International Conference on Computational Linguistics，2018：3272-3282.
[18] YIN W，YU M，XIANG B.Simple question answering by attentive convolutional neural network[J].arXiv：1606.
03391.2016.
[19] HU S，TAN Z，ZENG W.Entity linking via symmetrical attention-based neural network and entity structural features[J].Symmetry，2019，11（4）：453.
[20] DO P，PHAN T H.Developing a BERT based triple classification model using knowledge graph embedding for question answering system[J].Applied Intelligence，2022，52（1）：636-651.
[21] 赵畅，李慧颖.面向知识库问答的实体链接方法[J].中文信息学报，2019，33（11）：125-133.
ZHAO C，LI H Y.An entity linking approach for knowledge base question answering[J].Journal of Chinese Information Processing，2019，33（11）：125-133.
[22] LUO D，SU J，YU S.A BERT-based approach with relation-aware attention for knowledge base question answering[C]//2020 International Joint Conference on Neural Networks，2020：1-8.
[23] ALBAWI S，MOHAMMED T A，AL-ZAWI S.Understanding of aconvolutional neural network[C]//2017 International Conference on Engineering and Technology，2017：1-6.
[24] VENUGOPALAN J，TONG L，HASSANZADEH H R.Multimodal deep learning models for early detection of Alzheimer’s disease stage[J].Scientific Reports，2021，11（1）：1-13.
[25] WANG X，CAO W.Non-iterative approaches in training feed-forward neural networks and their applications[J].Soft Computing，2018，22（11）：3473-3476.
[26] LI Y，MU L，LI H.Automatic answer ranking based on sememe vector in KBQA[C]//2019 International Conference on Asian Language Processing，2019：273-278.
[27] 周博通，孙承杰，林磊，等.基于LSTM的大规模知识库自动问答[J].北京大学学报（自然科学版），2018，54（2）：286-292.
ZHOU B T，SUN C J，LIN L，et al.LSTM based question answering for large scale knowledge base[J].Acta Scientiarum Naturalium Universitatis Pekinensis，2018，54（2）：286-292.
[28] LAI Y，LIN Y，CHEN J.Open domain question answering system based on knowledge base[M]//Natural language understanding and intelligent applications.Berlin：Springer，2016：722-733.
[29] 张芳容，杨青.知识库问答系统中实体关系抽取方法研究[J].计算机工程与应用，2020，56（11）：219-224.
ZHANG F R，YANG Q.Research on entity relation extraction method in knowledge-based question answering[J].Computer Engineering and Applications 2020，56（11）：219-224.
[30] ZHOU G，XIE Z，YU Z.DFM：a parameter-shared deep fused model for knowledge base question answering[J].Information Sciences，2021，547：103-118.
[31] LIU A，HUANG Z，LU H.BB-KBQA：BERT-based knowledge base question answering[C]//China National Conference on Chinese Computational Linguistics.Cham：Springer，2019：81-92.
[32] 吴天波，刘露平，罗晓东，等.基于弱依赖信息的知识库问答方法[J].计算机工程，2021，47（6）：76-82.
WU T B，LIU L P，LUO X D，et al.Knowledge base question answering based on weak dependency information[J].Computer Engineering，2021，47（6）：76-82.