Bidirectional Attention Question Answering Model Combining Knowledge Representation Learning

doi:10.3778/j.issn.1002-8331.2103-0478

Abstract

Abstract:

Question Answering over Knowledge Graph（KGQA） is one of the research hotspots in the field of natural language processing and has received extensive attention in recent years. KGQA faces challenges such as multi-hop problems that need to combine multiple triples for reasoning and incomplete knowledge graphs. To solve these problems, a KR-BAT model which combines knowledge representation and bidirectional attention mechanism is proposed. It introduces knowledge representation learning to improve the global modeling ability and deals with incomplete knowledge graph; the bidirectional attention model captures the rich interactive information between candidate answers and questions, and give answers after analysis and reasoning. Experiments are conducted on the MetaQA dataset and compared with baseline models such as VRN, KV-MemNN, GraftNet. Results show that KR-BAT achieves very competitive performance on the complete knowledge graph, and is further improved than the baseline model on the incomplete knowledge graph.

Key words: knowledge graph, intelligent question answering, knowledge representation, attention

摘要：

知识图谱问答是自然语言处理领域的研究热点之一，近年来受到广泛的关注。知识图谱问答面临需要结合多条三元组进行推理的多跳问题以及知识图谱不完整等挑战，为解决这些问题，提出了一种融合知识表示学习的双向注意力模型（Bidirectional Attention model combining Knowledge Representation，KR-BAT）。引入知识表示学习以提高模型全局建模能力，应对知识图谱不完整的情况；使用双向注意力模型捕捉候选答案和问题间丰富的交互信息，经过分析推理给出答案。在MetaQA数据集上进行了实验，对比VRN、KV-MemNN、GraftNet等基准模型，在完整知识图谱上达到了非常有竞争力的性能，在不完整知识图谱上大幅度优于基准模型。

关键词: 知识图谱, 智能问答, 知识表示, 注意力

LU Qi, PAN Zhisong, XIE Jun. Bidirectional Attention Question Answering Model Combining Knowledge Representation Learning[J]. Computer Engineering and Applications, 2021, 57(23): 171-177.

卢琪，潘志松，谢钧. 融合知识表示学习的双向注意力问答模型[J]. 计算机工程与应用, 2021, 57(23): 171-177.

References

[1] 王智悦，于清，王楠，等.基于知识图谱的智能问答研究综述[J].计算机工程与应用，2020，56（23）：7-17.
WANG Z Y，YU Q，WANG N，et al.Survey of intelligent question answering research based on knowledge graph[J].Computer Engineering and Applications，2020，56（23）：7-17.
[2] BORDES A，CHOPRA S，WESTON J.Question answering with subgraph embeddings[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing，2014：615-620.
[3] HUANG X，ZHANG J，LI D，et al.Knowledge graph embedding based question answering[C]//Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining，2019：105-113.
[4] SUN H，DHINGRA B，ZAHEER M，et al.Open domain question answering using early fusion of knowledge bases and text[C]//Empirical Methods in Natural Language Processing，2018：4231-4242.
[5] SUN H，BEDRAX-WEISS T，COHEN W W.PullNet：open domain question answering with iterative retrieval on knowledge bases and text[C]//Proceedings of EMNLP-IJCNLP，2019：2380-2390.
[6] BAO J，DUAN N，YAN Z，et al.Constraint-based question answering with knowledge graph[C]//International Conference on Computational Linguistics，2016：2503-2514.
[7] SEO M，KEMBHAVI A，FARHADI A，et al.Bidirectional attention flow for machine comprehension[C]//5th International Conference on Learning Representations，2016.
[8] TURIAN J P，RATINOV L A，BENGIO Y.Word representations：a simple and general method for semi-supervised learning[C]//Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics，Uppsala，Sweden，2010：384-394.
[9] 刘知远，孙茂松，林衍凯，等.知识表示学习研究进展[J].计算机研究与发展，2016，53（2）：247-261.
LIU Z Y，SUN M S，LIN Y K，et al.Knowledge representation learning：a review[J].Journal of Computer Research and Development，2016，53（2）：247-261.
[10] YANG B，YIH W T，HE X，et al.Embedding entities and relations for learning and inference in knowledge bases[C]//3rd International Conference on Learning Representations，2014.
[11] TROUILLON T，WELBL J，RIEDEL S，et al.Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning，2016：2071-2080.
[12] MIKOLOV T，CHEN K，CORRADO G S，et al.Efficient estimation of word representations in vector space[C]//1st International Conference on Learning Representations，2013.
[13] BORDES A，USUNIER N，GARCIA DURAN A，et al.Translating embeddings for modeling multi-relational data[C]//Advances in Neural Information Processing Systems，2013：2787-2795.
[14] WANG Z，ZHANG J，FENG J，et al.Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2014，28（1）：1112-1119.
[15] LIN Y，LIU Z，SUN M，et al.Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2015：2181-2187.
[16] JI G，HE S，XU L，et al.Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing，2015：687-696.
[17] ZHANG Y，LIU K，HE S，et al.Question answering over knowledge base with neural attention combining global knowledge information[J].arXiv：1606.00979，2016.
[18] ALVAREZMELIS D，JAAKKOLA T S.Tree-structured decoding with doubly-recurrent neural networks[C]//5th International Conference on Learning Representations，2017.
[19] MOHAMMED S，PENG S，LIN J.Strong baselines for simple question answering over knowledge graphs with and without neural networks[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2018：291-296.
[20] YIH W，HE X，MEEK C.Semantic parsing for single-relation question answering[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics，2014：643-648.
[21] HAO Y，ZHANG Y，LIU K，et al.An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics，2017：221-231.
[22] LI D，WEI F，MING Z，et al.Question answering over freebase with multi-column convolutional neural networks[C]//Meeting of the Association for Computational Linguistics & International Joint Conference on Natural Language Processing，2015：260-269.
[23] SAXENA A，TRIPATHI A，TALUKDAR P.Improving multi-hop question answering over knowledge graphs using knowledge base embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics，2020：4498-4507.
[24] SCHUSTER M，PALIWAL K K.Bidirectional recurrent neural networks[J].IEEE Transactions on Signal Processing，1997，45（11）：2673-2681.
[25] PENNINGTON J，SOCHER R，MANNING C.Glove：global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing，2014：1532-1543.
[26] ZHANG Y，DAI H，KOZAREVA Z，et al.Variational reasoning for question answering with knowledge graph[C]//Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence，2018：6069-6076.
[27] BALAEVI I，ALLEN C，HOSPEDALES T M.TuckER：tensor factorization for knowledge graph completion[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing，2019：5184-5193.
[28] MILLER A H，FISCH A，DODGE J，et al.Key-value memory networks for directly reading documents[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing，2016：1400-1409.