Computer Engineering and Applications ›› 2024, Vol. 60 ›› Issue (16): 105-115.DOI: 10.3778/j.issn.1002-8331.2311-0079

• Pattern Recognition and Artificial Intelligence • Previous Articles     Next Articles

Multi-Label Text Classification Combining Bidirectional Attention and Contrast Enhancement Mechanism

LI Jiandong, FU Jia, LI Jiaqi   

  1. 1.College of Software, Liaoning Technical University, Huludao, Liaoning 125105, China
    2.College of Mining, Liaoning Technical University, Fuxin, Liaoning 123000, China
  • Online:2024-08-15 Published:2024-08-15

融合双向注意力和对比增强机制的多标签文本分类

李建东,傅佳,李佳琦   

  1. 1.辽宁工程技术大学 软件学院,辽宁 葫芦岛 125105
    2.辽宁工程技术大学 矿业学院,辽宁 阜新 123000

Abstract: A multi-label text classification model that integrates bidirectional attention and contrast enhancement mechanism is proposed to address the issues of missing semantic information during text sequence growth and ignoring rich knowledge from existing instances when predicting specific labels. Firstly, combining BERT word embedding, the CTransformer model is used to obtain the global dependency relationship and local structural information of the sequence. Simultaneously, bidirectional attention and label embedding are used to further generate final text and label representations, text information is interacted with label information to obtain more comprehensive semantic information. Then, a contrast enhancement mechanism is used for KNN instance retrieval, a multi-label contrastive learning objective is designed to make the model aware of the KNN classification process and improve the quality of retrieving neighbor instances during the inference process. Finally, the classifier performs text classification based on label representation and text representation. To evaluate the performance of the model, it is tested on three publicly available English datasets, and the experimental results show that the proposed model outperforms P@K and nDCG@K compared to other mainstream baseline models in terms of indicators.

Key words: multi-label text classification, attention mechanism, contrast enhancement mechanism, CTransformer model, multi-label comparative learning

摘要: 现有多标签文本分类模型面对文本序列增长时语义信息缺失,预测特定标签时忽略已有实例中的丰富知识等问题不能很好地解决,由此提出一种融合双向注意力和对比增强机制的多标签文本分类模型。结合BERT词嵌入,利用CTransformer模型分别获取序列的全局依赖关系和局部结构信息;使用双向注意力和标签嵌入进一步生成最终文本表示和标签表示,将文本信息与标签信息进行交互,得到更为全面的综合语义信息;利用对比增强机制进行KNN实例检索,设计一个多标签对比学习目标,使模型意识到KNN分类过程,并提高推理过程中检索邻居实例的质量。分类器根据标签表示和文本表示进行文本分类。为了评估模型的性能,在三个公开英文数据集上进行测试,实验结果表明提出的模型在P@K和nDCG@K指标上均优于其他主流基线模型。

关键词: 多标签文本分类, 注意力机制, 对比增强机制, CTransformer模型, 多标签对比学习