计算机工程与应用 ›› 2023, Vol. 59 ›› Issue (11): 80-87.DOI: 10.3778/j.issn.1002-8331.2206-0328

• 模式识别与人工智能 • 上一篇    下一篇

面向短文本语义分析的标签条件神经主题模型

王嫄,鄢艳玲,徐茂玲,胡鹏,赵婷婷,杨巨成   

  1. 1.天津科技大学 人工智能学院,天津 300457
    2.普迈康(天津)精准医疗科技有限公司,天津 300000
  • 出版日期:2023-06-01 发布日期:2023-06-01

Label-Conditional Neural Topic Model for Semantic Analysis of Short Texts

WANG Yuan, YAN Yanling, XU Maoling, HU Peng, ZHAO Tingting, YANG Jucheng   

  1. 1.College of Artificial Intelligence, Tianjin University of Science & Technology, Tianjin 300457, China
    2.Population and Precision Health Care, Ltd, Tianjin 300000, China
  • Online:2023-06-01 Published:2023-06-01

摘要: 无监督机器学习方法中的神经主题模型已被广泛用于自动挖掘文本潜在语义。然而,短文本篇幅有限,文中可用于推断的信息匮乏,模型难以在上下文不充分的情况下正确识别歧义词。为此,提出了一种面向短文本语义分析的标签条件神经主题模型,模型采用变分自编码器架构,在编码器输出的主题分布上引入文本的标签信息,作为主题类别级的语义标识符指导模型过滤与当前主题语义不相关的词、凝练语义并辨识歧义词在主题语境下的准确词义,引导模型推断离散一致的主题。针对短文本应用过程中主题语义分布统计显著有偏的数据特点,在模型训练过程中引入泰勒损失,通过调整泰勒多项式系数建模短文本类别分布不平衡。实验结果表明,该模型不仅能够极大提高短文本主题建模的质量,生成连贯且多样的主题,而且能有效提升下游任务性能。

关键词: 神经主题模型, 短文本, 泰勒损失

Abstract: Neural topic models in unsupervised machine learning methods have been widely used to automatically mine the text for latent semantics. However, the limited length of short text and the scarcity of information available for inference in the text makes it difficult for the model to correctly identify ambiguous words with insufficient context. Therefore, a label-conditional neural topic model for semantic analysis of short texts is proposed. The model adopts a variational auto-encoder architecture, which introduces the label information of the text as a semantic identifier at the topic category level on the topic distribution of the encoder output to guide the model to filter words that are not semantically relevant to the current topic, condense the semantics, and identify the exact word meanings of ambiguous words in the topic context to guide the model to infer discrete consistent topic. To address the data characteristics of statistically significant bias of topic semantic distribution during the application of short texts, PolyLoss is introduced in the model training process, and the imbalance of short text category distribution is modeled by adjusting Taylor polynomial coefficients. The experimental results show that the model can not only greatly improve the quality of short-text topic modeling, and generate coherent and diverse topics, but also effectively improve the performance of downstream tasks.

Key words: neural topic models, short texts, PolyLoss