Review of Conversational Machine Reading Comprehension for Knowledge Graph

doi:10.3778/j.issn.1002-8331.2305-0081

Abstract

Abstract: Conversational machine reading comprehension develops with the development of datasets. The purpose is to allow the machine to have multiple rounds of dialogue on the basis of understanding the content of the article. However, the existing model method cannot capture the most related historical information related to the current problem from the history of the dialogue. The inference capacity of the model is poor, and it is difficult to obtain the implicit information between the entities. The application of knowledge graph to reasoning question answering is a major research hotspot. Knowledge graph technology can infer the implicit relationship between entities, and when applied to reasoning questions and answering, it can improve the model’s reasoning question answering ability and improve the accuracy of prediction. In recent years, the widespread application of knowledge map reasoning technology has greatly promoted the development of knowledge map reasoning and answers. The conversational machine reading comprehension based on knowledge graph is summarized from three aspects. Firstly, it introduces the data sets in the field of session machine reading comprehension and some current model methods, and makes a brief analysis and comparison of the performance, advantages and disadvantages of the model. Then, it introduces the definition, architecture and four core technologies of the knowledge map, and briefly introduces the model methods of the three types of knowledge map reasoning quiz. Finally, the work is summarized, and according to the characteristics and knowledge of data sets and knowledge of the reading and comprehension of the session machine, the future research focus is prospected.

Key words: machine reading comprehension, multiple rounds of dialogue, knowledge graph, knowledge graph question answering

摘要： 对话式机器阅读理解随着数据集的发展而发展，目的在于让机器在理解文章内容的基础上能够进行多轮对话。但现有的模型方法无法从对话历史中捕获到与当前问题最相关的历史信息，模型的推理能力较差，很难获取实体间的隐含信息。知识图谱应用于推理问答是当前的一大研究热点。知识图谱技术可以推断出实体间的隐含关系，应用于推理问答则能够提升模型的推理问答能力，提高预测的准确率。近年来，知识图谱推理技术的广泛应用，极大地推动了知识图谱推理问答的发展。对基于知识图谱的会话式机器阅读理解从三方面进行总结：介绍了会话式机器阅读理解领域的数据集以及当前的一些典型的模型方法，并对模型的性能和优缺点作了简要的分析与比较；介绍了知识图谱的定义、架构以及四大核心技术，并简要介绍了三大类知识图谱推理问答的模型方法；最后总结工作，并根据会话式机器阅读理解的数据集特点和知识图谱推理问答模型的缺点，对未来的研究重点进行展望。

关键词: 机器阅读理解, 多轮对话, 知识图谱, 知识图谱推理问答

HU Juan, XI Xuefeng, CUI Zhiming. Review of Conversational Machine Reading Comprehension for Knowledge Graph[J]. Computer Engineering and Applications, 2024, 60(3): 17-28.

胡娟, 奚雪峰, 崔志明. 面向知识图谱的会话式机器阅读理解研究综述[J]. 计算机工程与应用, 2024, 60(3): 17-28.

References

[1] LEHNERT W G. The process of question and answering[M].New Haven: Yale University, 1977.
[2] REDDY S, CHEN D, MANNING C D. CoQA: a conversational question answering challenge[J]. Transactions of the Association for Computational Linguistics, 2019, 7: 249-266.
[3] CHOI E, HE H, IYYER M, et al. QuAC: question answering in context[J]. arXiv:1808.07036, 2018.
[4] LOVELACE J, NEWMAN-GRIFFIS D, VASHISHTH S, et al. Robust knowledge graph completion with stacked convolutions and a student re-ranking network[J]. arXiv:2106.06555, 2021.
[5] YATSKAR M. A qoalitative comparison of CoQA, squad 2.0 and quac[J]. arXiv:1809.10735, 2018.
[6] HERMANN K M, KOCISKY T, GREFENSTETTE E, et al. Teaching machines to read and comprehend[C]//Advances in Neural Information Processing Systems, 2015.
[7] RICHARDSON M, BURGES C J C, RENSHAW E. MCTest: a challenge dataset for the open-domain machine comprehension of text[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 2013: 193-203.
[8] LAI G, XIE Q, LIU H, et al. RACE: large-scale reading comprehension dataset from examinations[J]. arXiv:1704.04683, 2017.
[9] BELL J. Pragmatic reasoning: inferring contexts[C]//International and Interdisciplinary Conference on Modeling and Using Context. Berlin, Heidelberg: Springer, 1999: 42-53.
[10] QU C, YANG L, QIU M, et al. Attentive history selection for conversational question answering[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019: 1391-1400.
[11] JU Y, ZHAO F, CHEN S, et al. Technical report on conversational question answering[J]. arXiv:1909.10772, 2019.
[12] GAO J, GALLEY M, LI L. Neural approaches to conversational AI[C]//Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, 2018: 1371-1374.
[13] HUANG H Y, CHOI E, YIH W. FlowQA: grasping flow in history for conversational machine comprehension[J]. arXiv:1810.06683, 2018.
[14] DEVlIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[J]. arXiv:1810.04805, 2018.
[15] PENNINGTON J, SOCNER R, MANNING C D. GloVe: global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014: 1532-1543.
[16] ZHU C, ZENG M, HUANG X. SDNet: contextualized attention-based deep network for conversational question answering[J]. arXiv:1812.03593, 2018.
[17] CHEN D, FISCH A, WESTON J, et al. Reading wikipedia to answer open-domain questions[J]. arXiv:1704.00051, 2017.
[18] RAJPURKAR P, ZHANG J, LOPYREV K, et al. SQUAD: 100,000+ questions for machine comprehension of text[J]. arXiv:1606.05250, 2016.
[19] LABUTOV I, YANG B, PRAKASH A, et al. Multi-relational question answering from narratives: machine reading and reasoning in simulated worlds[J]. arXiv:1902.09093, 2019.
[20] SEE A, LIU P J, MANNING C D. Get to the point: summarization with pointer-generator networks[J]. arXiv:1704. 04368, 2017.
[21] SEO M, KEMBHAVI A, FARHADI A, et al. Bidirectional attention flow for machine comprehension[J]. arXiv:1611. 01603, 2016.
[22] YEH Y T, CHEN Y N. FlowDelta: modeling flow information gain in reasoning for conversational machine comprehension[J]. arXiv:1908.05117, 2019.
[23] CHEN Y, WU L, ZAKI M J. GraphFlow: exploiting conversation flow with graph neural networks for conversational machine comprehension[J]. arXiv:1908.00059, 2019.
[24] ZHANG X. MC2: multi-perspective convolutional cube for conversational machine reading comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 6185-6190.
[25] QU C, YANG L, QIU M, et al. BERT with history answer embedding for conversational question answering[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019: 1133-1136.
[26] OHSUGI Y, SAITO I, NISHIDA K, et al. A simple but effective method to incorporate multi-turn context with BERT for conversational machine comprehension[J]. arXiv:1905.12848, 2019.
[27] GONG H, SHEN Y, YU D, et al. Recurrent chunking mechanisms for long-text machine reading comprehension[J]. arXiv:2005.08056, 2020.
[28] ZHAO J, BAO J, WANG Y, et al. RoR: read-over-read for long document machine reading comprehension[J]. arXiv: 2109.04780, 2021.
[29] LIU Q, LI Y, YANG D H, et al. Knowledge graph construction techniques[J]. Journal of Computer Research and Development, 2016, 53(3): 582-600.
[30] DONG X, GABRILOVICH E, HEITZ G, et al. Knowledge vault: a web-scale approach to probabilistic knowledge fusion[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014: 601-610.
[31] BENGIO Y. Learning deep architectures for AI[J]. Foundations and Trends? in Machine Learning, 2009, 2(1): 1-127.
[32] BENGIO Y, COURVILLE A, VINENT P. Representation learning: a review and new perspectives[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8): 1798-1828.
[33] LIU Z Y, SUN M S, LIN Y K, et al. Knowledge representation learning: a review[J]. Journal of Computer Research and Development, 2016, 53(2): 1-16.
[34] XU X K, FANG D W, JIANG X, et al. Research on knowledge granularity representation and standardization during knowledge organization[J]. Document, Information & Knowledge, 2014(6): 101-106.
[35] STUDER R, BENJAMINS V R, FENSEl D. Knowledge engineering: principles and methods[J]. Data & Knowledge Engineering, 1998, 25(1/2): 161-197.
[36] WONG W, LIU W, BENNAMOUN M. Ontology learning from text: a look back and into the future[J]. ACM Computing Surveys (CSUR), 2012, 44(4): 1-36.
[37] WU W, LI H, WANG H, et al. Probase: a probabilistic taxonomy for text understanding[C]//Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, 2012: 481-492.
[38] FADER A, SODERIAND S, ETZIONI O. Identifying relations for open information extraction[C]//Proceedings of the 2011 Conference on Empirical Methods in natural Language Processing, 2011: 1535-1545.
[39] MDES P N, MUHLEISEN H, BIZER C. Sieve: linked data quality assessment and fusion[C]//Proceedings of the 2012 Joint EDBT/ICDT Workshops, 2012: 116-123.
[40] TAN C H, AGICHTEIN E, IPEIROTIS P, et al. Trust, but verify: predicting contribution quality for knowledge base construction and curation[C]//Proceedings of the 7th ACM International Conference on Web Search and Data Mining, 2014: 553-562.
[41] GENG X, ZHANG J J, LI W Y. Knowledge graph construction techniques[J]. Computer Science, 2014, 41(7): 148-152.
[42] RICHARDSON M, DOMINGOS P. Markov logic networks[J]. Machine Learning, 2006, 62(1): 107-136.
[43] GALARRAGA L A, TEFLIOUDI C, HOSE K, et al. AMIE: association rule mining under incomplete evidence in ontological knowledge bases[C]//Proceedings of the 22nd International Conference on World Wide Web,2013: 413-422.
[44] YIH W, HE X, MEEK C. Semantic parsing for single-relation question answering[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2014: 643-648.
[45] YIH S W, CHANG M W, HE X, et al. Semantic parsing via staged query graph generation: Question answering with knowledge base[C]//Proceedings of the Joint Conference of the 53rd Annual Meeting of the ACL and the 7th International Joint Conference on Natural Language Processing of the AFNLP, 2015.
[46] LUO K, LIN F, LUO X, et al. Knowledge base question answering via encoding of complex query graphs[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018: 2185-2194.
[47] SOROKIN D, GUREVYCH I. Modeling semantics with gated graph neural networks for knowledge base question answering[J]. arXiv:1808.04126, 2018.
[48] MAHESHWARI G, TRIVEDI P, LUKOVNIKOV D, et al. Learning to rank query graphs for complex question answering over knowledge graphs[C]//Proceedings of the International Semantic Web Conference. Cham: Springer, 2019: 487-504.
[49] HU S, ZOU L, ZHANG X. A state-transition framework to answer complex questions over knowledge base[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018: 2098-2108.
[50] DING J, HU W, XU Q, et al. Leveraging frequent query substructures to generate formal queries for complex question answering[J]. arXiv:1908.11053, 2019.
[51] LAN Y, JIANG J. Query graph generation for answering multi-hop complex questions from knowledge bases[C]//Proceedings of the Association for Computational Linguistics, 2020.
[52] CHEN Y, LI H, HUA Y, et al. Formal query building with query structure prediction for complex question answering over knowledge base[J]. arXiv:2109.03614, 2021.
[53] HAMILTON W, BAJAJ P, ZITNIK M, et al. Embedding logical queries on knowledge graphs[C]//Advances in Neural Information Processing Systems, 2018.
[54] BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Advances in Neural Information Processing Systems, 2013.
[55] TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the International Conference on Machine Learning, 2016: 2071-2080.
[56] HUANG X, ZHANG J, LI D, et al. Knowledge graph embedding based question answering[C]//Proceedings of the 12th ACM International Conference on Web Search and Data Mining, 2019: 105-113.
[57] WANG M, WANG R, LIU J, et al. Towards empty answers in SPARQL: approximating querying with RDF embedding[C]//Proceedings of the 17th International Semantic Web Conference (ISWC 2018), Monterey, CA, USA, October 8-12, 2018: 513-529.
[58] WANG R, WANG M, LIU J, et al. Leveraging knowledge graph embeddings for natural language question answering[C]//Proceedings of the 24th International Conference on Database Systems for Advanced Applications, Chiang Mai, Thailand, April 22-25, 2019: 659-675.
[59] SUN H, ARNOLD A, BEDRAX WEISS T, et al. Faithful embeddings for knowledge base queries[C]//Advances in Neural Information Processing Systems, 2020: 22505-22516.
[60] SAXENA A, TRIPATHI A, TALUKDAR P. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 4498-4507.
[61] NEELAKANTAN A, ROTH B, MCCALLUM A. Compositional vector space models for knowledge base completion[J]. arXiv:1504.06662, 2015.
[62] LIU Y, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized bert pretraining approach[J]. arXiv:1907.11692, 2019.
[63] DAS R, NEELAKANTAN A, BELANGER D, et al. Chains of reasoning over entities, relations, and text using recurrent neural networks[J]. arXiv:1607.01426, 2016.
[64] ZHANG Y, DAI H, KOZAREVA Z, et al. Variational reasoning for question answering with knowledge graph[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2018.
[65] SUN H, DHINGRA B, ZAHEER M, et al. Open domain question answering using early fusion of knowledge bases and text[J]. arXiv:1809.00782, 2018.
[66] SUN H, BEDRAX-WEISS T, COHEN W W. PullNet: open domain question answering with iterative retrieval on knowledge bases and text[J]. arXiv:1904.09537, 2019.
[67] XIONG W, YU M, CHANG S, et al. Improving question answering over incomplete KBs with knowledge-aware reader[J]. arXiv:1905.07098, 2019.
[68] DAS R, DHULIAWALA S, ZAHEER M, et al. Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning[J]. arXiv:1711.05851, 2017.
[69] LIN X V, SOCHER R, XIONG C. Multi-hop knowledge graph reasoning with reward shaping[J]. arXiv:1808.10568, 2018.
[70] QIU Y, WANG Y, JIN X, et al. Stepwise reasoning for multi-relation question answering over knowledge graph with weak supervision[C]//Proceedings of the 13th International Conference on Web Search and Data Mining, 2020: 474-482.
[71] HE G, LAN Y, JIANG J, et al. Improving multi-hop knowledge base question answering by learning intermediate supervision signals[C]//Proceedings of the 14th ACM International Conference on Web Search and Data Mining, 2021: 553-561.
[72] GAO L, ZHAO W, ZHANG J, et al. G2S: semantic segment based semantic parsing for question answering over knowledge graph[J]. Acta Electonica Sinica, 2021, 49(6): 1132.
[73] WANG Z, ZHANG J, FENG J, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2014.
[74] YAN J, RAMAN M, CHAN A, et al. Learning contextualized knowledge structures for commonsense reasoning[J]. arXiv:2010.12873, 2020.