改进卷积神经网络的文本主题识别算法研究

doi:10.3778/j.issn.1002-8331.2007-0500

摘要/Abstract

摘要： 针对于传统方法中存在的文本特征表示能力差、模型主题识别准确率低等问题，提出一种融合SENet与卷积神经网络的文本主题识别方法。将每个词对应的Word2vec词向量与LDA主题向量进行融合，并依据词语对主题的贡献度完成文档加权向量化处理；构建SECNN主题识别模型，使用SENet对卷积层输出的特征图进行权值的重标定，依靠其提升重要特征并抑制无用特征的性能，高效地进行主题识别；使用FDA评估样本的类别表征能力，提出FDA-SGD算法对模型参数进行调优，完成文本主题识别任务。使用新闻文本数据集验证改进算法的有效性，通过与传统模型对比表明，改进算法可以有效提高模型的收敛速度，具有较好的主题识别能力。

关键词: 主题识别, SENet, 卷积神经网络, Word2vec, 隐含狄利克雷分布（LDA）

Abstract: Aiming at the problems of poor text feature representation ability and low model topic recognition accuracy in traditional methods, a text topic recognition method combining SENet and convolutional neural network is proposed. First, the Word2vec word vector corresponding to each word is fused with the LDA topic vector, and the document weighted vectorization process is completed according to the contribution of the word to the topic. Then the SECNN topic recognition model is constructed, and the SENet is used to perform the feature map output of the convolutional layer. The recalibration of weights relies on the performance of improving important features and suppressing useless features to efficiently perform topic identification. Finally, the FDA is used to evaluate the category representation ability of the sample, and the FDA-SGD algorithm is proposed to optimize the model parameters and complete the text topic recognition task. The news text data set is used to verify the effectiveness of the improved algorithm. The comparison with the traditional model shows that the improved algorithm can effectively improve the convergence speed of the model and has better topic recognition capabilities.

Key words: topic recognition, SENet, convolutional neural network, Word2vec, latent Dirichlet allocation（LDA）

邱宁佳, 杨长庚, 王鹏, 任涛. 改进卷积神经网络的文本主题识别算法研究[J]. 计算机工程与应用, 2022, 58(2): 161-168.

QIU Ningjia, YANG Changgeng, WANG Peng, REN Tao. Research on Text Topic Recognition Algorithm Based on Improved Convolutional Neural Network[J]. Computer Engineering and Applications, 2022, 58(2): 161-168.

参考文献

[1] BLEI D M，NG A Y，JORDAN M I.Latent Dirichlet allocation[J].The Journal of Machine Learning Research，2003，3（4/5）：993-1022.
[2] MIKOLOV T，CHEN K，CORRADO G S，et al.Efficient estimation of word representations in vector space[EB/OL].[2020-07-29].https：//arxiv.org/abs/1301.3781.
[3] 陈培新，郭武.融合潜在主题信息和卷积语义特征的文本主题分类[J].信号处理，2017，33（8）：1090-1096.
CHEN Peixin，GUO Wu.Document topic categorization combining latent topic information and convolutional semantic features[J].Journal of Signal Processing，2017，33（8）：1090-1096.
[4] 陈文实，刘心惠，鲁明羽.面向多标签文本分类的深度主题特征提取[J].模式识别与人工智能，2019，32（9）：785-792.
CHEN Wenshi，LIU Xinhui，LU Mingyu.Feature extraction of deep topic model for multi-label text classification[J].Pattern Recognition and Artificial Intelligence，2019，32（9）：785-792.
[5] 刘心惠，陈文实，周爱，等.基于联合模型的多标签文本分类研究[J].计算机工程与应用，2020，56（14）：111-117.
LIU Xinhui，CHEN Wenshi，ZHOU Ai，et al.Multi-label text classification based on joint model[J].Computer Engineering and Applications，2020，56（14）：111-117.
[6] GU Y，GU M，LONG Y，et al.An enhanced short text categorization model with deep abundant representation[J].World Wide Web，2018，21（6）：1705-1719.
[7] ZHOU W，WANG H，SUN H，et al.A method of short text representation based on the feature probability embedded vector[J].Sensors，2019，19（17）：3728.
[8] KIM Y.Convolutional neural networks for sentence classification[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing.Stroudsburg，USA：Associationfor Computational Linguistics，2014：1746-1751.
[9] TAO W，CHANG D.News text classification based on an improved convolutional neural network[J].Tehnicki Vjesnik-technical Gazette，2019，26（5）：1400-1409.
[10] BAKER S，KORHONEN A，PYYSALO S.Cancer hallmark text classification using convolutional neural networks[EB/OL].[2020-07-29].https：//www.aclweb.org/anthology/W16-5101.pdf.
[11] 曾凡锋，李玉珂，肖珂.基于卷积神经网络的语句级新闻分类算法[J].计算机工程与设计，2020，41（4）：978-982.
ZENG Fanfeng，LI Yuke，XIAO Ke.Sentence-level fine-grained news classification based on convolutional neural network[J].Computer Engineering and Design，2020，41（4）：978-982.
[12] ZHOU C，SUN C，LIU Z，et al.A C-LSTM neural network for text classification[J].Computer Science，2015，1（4）：39-44.
[13] 王海涛，宋文，王辉.一种基于LSTM和CNN混合模型的文本分类方法[J].小型微型计算机系统，2020，41（6）：1163-1168.
WANG Haitao，SONG Wen，WANG Hui.Text classification method based on hybrid model of LSTM and CNN[J].Journal of Chinese Computer Systems，2020，41（6）：1163-1168.
[14] 蔡林森，彭超，陈思远，等.基于多样化特征卷积神经网络的情感分析[J].计算机工程，2019，45（4）：169-174.
CAI Linsen，PENG Chao，CHEN Siyuan，et al.Sentiment analysis based on multiple features convolutional neural networks[J].Computer Engineering，2019，45（4）：169-174.
[15] 赖文辉，乔宇鹏.基于词向量和卷积神经网络的垃圾短信识别方法[J].计算机应用，2018，38（9）：2469-2476.
LAI Wenhui，QIAO Yupeng.Spam messages recognizing method based on word embedding and convolutional neural network[J].Journal of Computer Application，2018，38（9）：2469-2476.
[16] HU J，SHEN L，SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington DC，USA：IEEE Computer Society，2018：7132-7141.
[17] 周婉婷.基于词向量的文本倾向分析的向量表示方法研究[D].长春：东北师范大学，2019.
ZHOU Wanting.Research on vector representation of text sentiment analysis based on word vectors[D].Changchun：Northeast Normal University，2019.