End-to-End Aspect-Based Sentiment Analysis Model for BERT and LSI

doi:10.3778/j.issn.1002-8331.2303-0220

Abstract

Abstract: A model LSI-BERT based on BERT and fused lexical and syntactic information (LSI) is proposed to address the shortcomings of the existing end-to-end aspect-based sentiment analysis (E2E-ABSA) method research that does not fully utilize textual information. A BERT embedding layer and a TFM feature extractor are used to extract semantic information, and lexical information is extracted by the industrial-grade natural language processing tool SpaCy. Two weighting factors α and β are introduced to fuse semantic and lexical information. Graph attention networks (GAT) is used to extract syntactic dependency information based on the adjacency matrix generated from the syntactic dependency tree. A dual-stream attention network is used to fuse syntactic dependency information and textual information fused with lexical information to achieve better interaction between these two types of information. The experimental results show that the model outperforms the current representative model on three commonly used benchmark datasets.

Key words: end-to-end, aspect-based sentiment analysis, graph attention networks (GAT), weighting factors, dual-stream attention networks

摘要： 针对现有基于端到端方面的情感分析（E2E-ABSA）方法研究中没有充分利用文本信息的不足，提出了一种基于BERT与融合词性、句法信息（lexical and syntactic information，LSI）的模型LSI-BERT。使用BERT嵌入层和TFM特征提取器来提取语义信息，并通过工业级自然语言处理工具SpaCy提取词性信息，引入两个权重因子α和β对语义与词性信息进行融合；采用图注意网络（graph attention networks，GAT）根据句法依存树生成的邻接矩阵进行句法依存信息的提取；利用双流注意力网络针对句法依存信息和融合了词性信息的文本信息进行融合，使这两种信息实现更好的交互。实验结果表明，模型在三个常用基准数据集上的性能优于当前代表模型。

关键词: 端到端, 基于方面的情感分析, 图注意网络, 权重因子, 双流注意力网络

DAI Jiamei, KONG Weiwei, WANG Ze, LI Peizhe. End-to-End Aspect-Based Sentiment Analysis Model for BERT and LSI[J]. Computer Engineering and Applications, 2024, 60(12): 144-152.

代佳梅, 孔韦韦, 王泽, 李佩哲. BERT和LSI的端到端方面级情感分析模型[J]. 计算机工程与应用, 2024, 60(12): 144-152.

References

[1] ARWA S F, MOSTAFA E S, OSAMA A A. Arabic aspect extraction based on stacked contextualized embedding with deep learning[J]. IEEE Access, 2022, 10: 30526-30535.
[2] LIU N, SHEN B. Aspect term extraction via information-augmented neural network[J]. Complex & Intelligent Systems, 2023, 9(1): 537-563.
[3] 赵丽华, 王春立, 初钰凤. 基于注意力双层BiReGU模型的方面术语提取方法[J]. 计算机工程与应用, 2021, 57(22): 160-165.
ZHAO L H, WANG C L, CHU Y F. Attention-based double BiReGU model for aspect term extraction[J]. Computer Engineering and Applications, 2021, 57(22): 160-165.
[4] WANG Q L, WEN Z Y, ZHAO Q, et al. Progressive self-training with discriminator for aspect term extraction[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), Punta Cana, Dominican Republic, November 7-11, 2021. Stroudsburg: ACL, 2021: 257-268.
[5] CHEN Z, QIAN T. Transfer capsule network for aspect level sentiment classification[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), Florence, Italy, July 28-August 2, 2019. Stroudsburg: ACL, 2019: 547-556.
[6] 杨春霞, 宋金剑, 姚思诚. 面向方面级情感分析的加权依存树卷积网络[J]. 中文信息学报, 2022, 36(5): 125-132.
YANG C X, SONG J J, YAO S C. A weighted dependency tree convolutional networks for aspect-based sentiment analysis[J]. Journal of Chinese Information Processing, 2022, 36(5): 125-132.
[7] 王泽, 孔韦韦, 薛佳伟, 等. 基于 AWI和GCN的方面级情感分类模型[J]. 计算机工程与应用, 2023, 59(3): 135-142.
WANG Z, KONG W W, XUE J W, et al. Aspect level sentiment classification model based on AWI and GCN[J]. Computer Engineering and Applications, 2023, 59(3): 135-142.
[8] MA D, LI S, WU F, et al. Exploring sequence-to-sequence learning in aspect term extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), Florence, July 28-August 2, 2019. Stroudsburg: ACL, 2019: 3538-3547.
[9] ZHANG W, LI X, DENG Y, et al. A survey on aspect-based sentiment analysis: tasks, methods, and challenges[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 14(8): 1-20.
[10] LI K, CHEN C, QUAN X, et al. Conditional augmentation for aspect term extraction via masked sequence-to-sequence generation[C]//Proceedings of the 58th Annual Meeting of the Association for Computional Linguistics (ACL). Stroudsburg: ACL, 2020: 7056-7066.
[11] CHEN Z, QIAN T. Bridge-based active domain adaptation for aspect term extraction[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Stroudsburg: ACL, 2021: 317-327.
[12] KLEIN A, PEREG O, KORAT D, et al. Opinion-based relational pivoting for cross-domain aspect term extraction[C]//Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, Dublin, Ireland, May 23-25, 2022. Stroudsburg: ACL, 2022: 104-112.
[13] TANG J, LU Z, SU J, et al. Progressive self-supervised attention learning for aspect-level sentiment analysis[C]//Proceeding of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), Florence, Italy, July 28-August 2, 2019. Stroudsburg: ACL, 2019: 557-566.
[14] TANG H, JI D, LI C, et al. Dependency graph enhanced dual-transformer structure for aspect-based sentiment classification[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). Stroudsburg: ACL, 2020: 6578-6588.
[15] MA F, ZHANG C, SONG D. Exploiting position bias for robust aspect sentiment classification[J]. arXiv:2105.14210, 2021.
[16] 王泽, 孔韦韦, 黄纪云, 等. 基于LaLI和GCN的方面级情感分类模型[J]. 计算机工程与应用, 2023, 59(13): 110-119.
WANG Z, KONG W W, HUANG J Y, et al. Aspect level sentiment classification model based on LaLI and GCN[J]. Computer Engineering and Applications, 2023, 59(13): 110-119.
[17] LUO H, LI T, LIU B, et al. DOER: dual cross-shared RNN for aspect term-polarity co-extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), Florence, Italy, July 28-August 2, 2019. Stroudsburg: ACL, 2019: 591-601.
[18] HE R, LEE W S, NG H T, et al. An interactive multi-task learning network for end-to-end aspect-based sentiment analysis[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), Florence, Italy, July 28-August 2, 2019. Stroudsburg: ACL, 2019: 504-515.
[19] LUO H, JI L, LI T, et al. GRACE: gradient harmonized and cascaded labeling for aspect-based sentiment analysis[J]. arXiv:2009.10557, 2020.
[20] LIANG Y, MENG F, ZHANG J, et al. An iterative multi-knowledge transfer network for aspect-based sentiment analysis[J]. arXiv:2004.01935, 2020.
[21] LI X, BING L, ZHANG W, et al. Exploiting BERT for end-to-end aspect-based sentiment analysis[C]//Proceedings of the 5th Workshop on Noisy User-Generated Text (W-NUT 2019), Hong Kong, China, November 3-7, 2019. Stroudsburg: ACL, 2019: 34-41.
[22] LI X, BING L D, LI P J, et al. A unified model for opinion target extraction and target sentiment prediction[C]//Proceedings of the AAAI Conference on Artificial Intelligence, Hawaii, January 27-February 1, 2019. Menlo Park: AAAI, 2019: 6714-6721.
[23] CHEN Z, QIAN T. Relation-aware collaborative learning for unified aspect-based sentiment analysis[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020: 3685-3694.
[24] LIANG Y, MENG F, ZHANG J, et al. A dependency syntactic knowledge augmented interactive architecture for end-to-end aspect-based sentiment analysis[J]. Neurocomputing, 2021, 454(7): 291-302.
[25] LU T, XIANG Y, ZHANG L, et al. Sentence constituent-aware attention mechanism for end-to-end aspect-based sentiment analysis[J]. Multimedia Tools and Applications, 2022, 81(11): 15333-15348.
[26] BIE Y, YANG Y, ZHANG Y. Fusing syntactic structure information and lexical semantic information for end-to-end aspect-based sentiment analysis[J]. Tsinghua Science and Technology, 2022, 28(2): 230-243.
[27] LIU S, LI W, WU Y, et al. Jointly modeling aspect and sentiment with dynamic heterogeneous graph neural networks[J]. arXiv:2004.06427, 2020.