计算机工程与应用 ›› 2024, Vol. 60 ›› Issue (9): 203-211.DOI: 10.3778/j.issn.1002-8331.2212-0363

• 模式识别与人工智能 • 上一篇    下一篇

多层级信息增强异构图的篇章级话题分割模型

张洋宁,朱静,董瑞,尤泽顺,王震   

  1. 1.新疆农业大学 计算机与信息工程学院,乌鲁木齐 830052
    2.中国科学院 新疆理化技术研究所,乌鲁木齐 830011
    3.中国科学院大学,北京 100049
  • 出版日期:2024-05-01 发布日期:2024-04-29

Discourse-Level Topic Segmentation Model with Multi-Level Information Enhanced Heterogeneous Graphs Network

ZHANG Yangning, ZHU Jing, DONG Rui, YOU Zeshun, WANG Zhen   

  1. 1.College of Computer and Information Engineering, Xinjiang Agricultural University, Urumqi  830052, China
    2.Xinjiang Technical Institute of Physics and Chemistry, Chinese Academy of Sciences, Urumqi  830011, China
    3.University of Chinese Academy of Sciences, Beijing 100049, China
  • Online:2024-05-01 Published:2024-04-29

摘要: 话题分割是自然语言处理领域的基础任务之一,按照话题相关性原则将文本分割为多个话题相关的文本块。针对现有话题分割模型提取句子深层语义信息方面明显不足,并且忽略了篇章中的层次信息和上下文交互等问题,提出了一种多层级信息增强异构图的篇章级话题分割模型MHG-TS。该方法利用篇章中的句子和关键词构建异构图网络,引入BERT预训练语言模型捕获图中节点的深层语义特征,在句子节点一阶邻域层级,利用图注意力机制为语义关联的节点分配更大的边权重,增强了一阶邻域中语义关联节点的信息交互;在关键词节点层级,引入关键词信息加强句子语义特征表示;在句子高阶邻域层级,利用关键词节点作为中介,构建了句子节点高阶邻域中的跨句信息交互,丰富了句子节点之间的非序列关系,最终通过融合多层级信息实现包含全局语义信息的句子表示。相较于当下流行的模型,在多个数据集上,三个评价指标性能平均值分别提高了3.08%、2.56%、5.92%,取得了最佳的实验结果。

关键词: 图注意力机制, 预训练语言模型, 话题分割, 句子表示

Abstract: Topic segmentation is a basic task in the field of natural language processing, which divides the text into several semantically related text blocks according to the principle of semantic correlation. Nevertheless, the existing topic segmentation models are insufficient to extract the deep semantic information of sentences and further ignore the hierarchical information and contextual interaction in the discourse. To solve the above problems, this paper proposes a discourse-level topic segmentation model MHG-TS that enhances heterogeneous graphs through the multi-level information. MHG-TS constructs the network of heterogeneous graphs from the sentences and keywords in the discourse, adopts the pre-trained language model BERT to capture the deep semantic features of the nodes in the graph. At the level of first-order neighborhood, the model uses the graph attention mechanism to assign more weight to the semantic association nodes, which enhances the information interaction of semantic association nodes in the first-order neighborhood. At the level of keyword nodes, it adopts the information of keywords to enforce the representation of semantic features. At the level of high-order neighborhood, it adopts the keyword nodes as intermediaries to build the cross-sentence information interaction in the high-order neighborhood and to enrich the non-sequential relationship between sentence nodes, thus the sentence representations containing global semantic information is realized finally by integrating with multi-level information. Compared with the state-of-the-art model, the average values of MHG-TS’s performance of three evaluation indexes on many datasets increase by 3.08%, 2.56% and 5.92% respectively and the best experimental effects are obtained.

Key words: graph attention mechanism, pre-trained language model, topic segmentation, sentence encoding