Research on Construction of Tourism Knowledge Graph Integrating BERT-WWM and Pointer Network

doi:10.3778/j.issn.1002-8331.2111-0272

Abstract

Abstract: Aiming at the problems of scattered, disordered, and weak relevance of tourism information, a joint entity relationship extraction model integrating BERT-WWM（BERT with whole word masking） and pointer network is proposed to construct a tourism knowledge graph. With the help of the BERT-WWM pre-training language model, a sentence code containing a priori semantic knowledge is obtained from the crawled travel reviews. Because of the problems of error propagation, entity redundancy, and lack of interaction in the traditional entity relationship extraction method, as well as the characteristics of polysemy and overlapping relationship in tourism comments, it is proposed to directly model the triplet, extract the header entity by sentence coding, extract the tail entity according to the relationship category, and establish a cascade structure and pointer network to decode the output triplet. A tourism knowledge graph based on Neo4j graph database storage triples is built. The experiment is carried out on the eatablished tourism data set, the accuracy, recall, F1 value of the entity-relationship joint extraction model integrating BERT-WWM and pointer network are 93.42%, 86.59%, and 89.88%, respectively. Compared with the existing models, the three indicators show advantages. The constructed tourism knowledge graph realizes the integration and storage of scenic spot information. It has a particular practical reference significance for further promoting the development of the tourism industry.

Key words: BERT with whole word masking（BERT-WWM）, pointer network, tourism knowledge graph, relationship overlap, joint extraction of entity and relation

摘要： 针对旅游信息呈现出散乱、无序和关联性不强的问题，提出一种融合BERT-WWM（BERT with whole word masking）和指针网络的实体关系联合抽取模型构建旅游知识图谱。借助BERT-WWM预训练语言模型从爬取的旅游评论中获得含有先验语义知识的句子编码。针对传统的实体关系抽取方法存在错误传播、实体冗余、交互缺失等问题，以及旅游评论中的实体关系存在一词多义、关系重叠等特征，提出直接对三元组建模，利用句子编码抽取头实体，根据关系类别抽取尾实体，并建立级联结构和指针网络解码输出三元组。基于Neo4j图数据库存储三元组构建旅游知识图谱。实验在建立的旅游数据集上进行，融合BERT-WWM与指针网络的实体关系联合抽取模型的准确率、召回率和F1值分别为93.42%、86.59%和89.88%，与现有模型相比三项指标均显示出优越性，验证了该方法进行实体关系联合抽取的有效性。构建的旅游知识图谱实现了旅游景区信息的整合与存储，对进一步促进旅游业发展具有一定的实际参考意义。

关键词: BERT-WWM, 指针网络, 旅游知识图谱, 关系重叠, 实体关系联合抽取

XU Chun, LI Shengnan. Research on Construction of Tourism Knowledge Graph Integrating BERT-WWM and Pointer Network[J]. Computer Engineering and Applications, 2022, 58(12): 280-288.

徐春, 李胜楠. 融合BERT-WWM和指针网络的旅游知识图谱构建研究[J]. 计算机工程与应用, 2022, 58(12): 280-288.

References

[1] 徐紫嫣.旅游业融入“双循环”新发展格局：实施路径与政策思路[J].企业经济，2021，40（10）：143-150.
XU Z Y.Integrating tourism industry into new development pattern of “dual circulation”：implementation path and policy ideas[J].Enterprise Economy，2021，40（10）：143-150.
[2] 徐增林，盛泳潘，贺丽荣，等.知识图谱技术综述[J].电子科技大学学报，2016，45（4）：589-606.
XU Z L，SHENG Y P，HE L R，et al.Review on knowledge graph techniques[J].Journal of University of Electronic Science and Technology of China，2016，45（4）：589-606.
[3] 冯小兰，赵小兵.汉藏双语旅游领域知识图谱系统构建[J].中文信息学报，2019，33（11）：64-72.
FENG X L，ZHAO X B.A Chinese-Tibertan bilingual knowledge graph system in tourism domain[J].Journal of Chinese Information Processing，2019，33（11）：64-72.
[4] 曹明辉.面向旅游服务的智能标注及知识图谱构建系统[D].北京：北京邮电大学，2020.
CAO M H.Intelligent annotation and knowledge graph construction system for tourism service[D].Beijing：Beijing University of Posts and Telecommunications，2020.
[5] 韩凌洁.内蒙古自治区旅游领域知识库构建及应用研究[D].呼和浩特：内蒙古大学，2019.
HAN L J.Research on construction and application of ontology based knowledge base in Inner Mongolia autonomous region’s tourism domain[D].Hohhot：Inner Mongolia University，2019.
[6] 吴杰.以事件为中心的旅游知识图谱的构建与应用[D].北京：北京邮电大学，2021.
WU J.Event-centric tourism knowledge garph[D].Beijing：Beijing University of Posts and Telecommunications，2021.
[7] 陈荣祯.智慧旅游知识图谱的自动化构建研究及应用[D].北京：北京邮电大学，2021.
CHEN R Z.Research and application on automated construction of knowledge graph for smart trip[D].Beijing：Beijing University of Posts and Telecommunications，2021.
[8] 崔丽平，古丽拉·阿东别克，王智悦.基于有向图模型的旅游领域命名实体识别[J].计算机工程，2022，48（2）：306-313.
CUI L P，GULILA A，WANG Z Y.Named entity recognition in tourism based on direction graph model[J].Computer Engineering，2022，48（2）：306-313.
[9] 耿雪来.面向旅游的蒙汉双语知识图谱的研究与构建[D].呼和浩特：内蒙古大学，2019.
GENG X L.Research and construction of Mongolain-Chinese bilingual knowledge graph for tourism[D].Hohhot：Inner Mongolia University，2019.
[10] 张诺.面向山西旅游的知识图谱构建方法研究[D].太原：山西大学，2021.
ZHANG N.Research on knowledge graph construction oriented to Shanxi tourism[D].Taiyuan：Shanxi University，2021.
[11] MIWA M，BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Berlin：Association for Computational Linguistics，2016：1105-1116.
[12] ZHENG S，WANG F，BAO H，et al.Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Stroudsburg：Association for Computational Linguistics，2017：1227-1236.
[13] ZENG X，ZENG D，HE S，et al.Extracting relational facts by an end to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Stroudsburg：Association for Computational Linguistics，2018：506-514.
[14] 吴赛赛，梁晓贺，谢能付，等.面向领域实体关系联合抽取的标注方法[J].计算机应用，2021，41（10）：2858-2863.
WU S S，LIANG X H，XIE N F，et al.Annotation method for joint extraction of domain-oriented entity and relation[J].Journal of Computer Applications，2021，41（10）：2858-2863.
[15] CUI Y，CHE W，LIU T，et al.Pre-training with whole word masking for Chinese BERT[J].arXiv：1906.08101，2019.
[16] 黄泽民，吴晓鸰，吴迎岗，等.结合BERT和BiSRU-AT的中文文本情感分类[J].计算机工程与科学，2021，43（9）：1668-1675.
HUANG Z M，WU X L，WU Y G，et al.Analysis of Chinese text emotions combining BERT and BiSUR-AT[J].Computer Engineering & Science，2021，43（9）：1668-1675.
[17] 郭军成，万刚，胡欣杰，等.基于BERT的中文简历命名实体识别[J].计算机应用，2021，41（S1）：15-19.
GUO J C，WAN G，HU X J，et al.Chinese resume named entity recognition based on BERT[J].Journal of Computer Applications，2021，41（S1）：15-19.
[18] 唐焕玲，王慧，隗昊，等.面向时钟领域的BERT-LCRF命名实体识别方法[J/OL].计算机工程与应用：1-11（2021-06-22）[2021-11-03].http：//kns.cnki.net/kcms/detail/11.2127.
TP.20210622.1045.016.html.
TANG H L，WANG H，WEI H，et al.BERT-LCRF named entity recognition method oriented clock domain[J/OL].Computer Engineering and Applications：1-11（2021-06-22）[2021-11-03].http：//kns.cnki.net/kcms/detail/11.2127.TP.
20210622.1045.016.html.
[19] 王子牛，姜猛，高建瓴，等.基于BERT的中文命名实体识别方法[J].计算机科学，2019，46（11）：138-142.
WANG Z N，JIANG M，GAO J L，et al.Chinese named entity recognition method based on BERT[J].Computer Science，2019，46（11）：138-142.
[20] 马建红，魏字默，陈亚萌.基于信息融合标注的实体及关系联合抽取方法[J].计算机应用与软件，2021，38（7）：159-166.
MA J H，WEI Z M，CHEN Y M，et al.Entity and relationship joint extraction method based on information fusion tagging[J].Computer Applications and Software，2021，38（7）：159-166.
[21] 胡滨，汤保虎，姜海燕，等.家禽诊疗文本多实体关系联合抽取模型研究[J].农业机械学报，2021，52（6）：268-276.
HU B，TANG B H，JIANG H Y，et al.Joint extraction model of multi-entity relations for poultry diagnosis and treatment text[J].Transactions of the Chinese Society for Agricultural Machinery，2021，52（6）：268-276.