Attention-Based Double BiReGU Model for Aspect Term Extraction

doi:10.3778/j.issn.1002-8331.2007-0038

Abstract

Abstract:

In aspect-based sentiment analysis, aspect term extraction is an important task, which aims to extract key aspect term from online product reviews. To solve the problem of aspect term extraction, attention-based double BiReGU model is proposed. Based on the traditional BiLSTM, this model introduces double embedding mechanism and ReGU（Residual Gated Unit） as auxiliary to improve the ability of feature extraction. Firstly, BiReGU is used to learn textual feature representation to capture the long-term dependence between words better. Secondly, the attention mechanism is introduced after the first layer BiReGU, and different weights are assigned to each word to obtain the new knowledge representation after the fusion of feature, which is input to the second layer BiReGU to learn more global representation of text feature. Finally, the task of extracting aspect term is realized. The experimental results on Laptop and Restaurant datasets in the classic SemEval 2014 demonstrate the effectiveness of the proposed model.

Key words: attention mechanism, aspect term extraction, BiReGU, aspect-based sentiment analysis, deep learning

摘要：

方面术语提取是方面级情感分析中的一项重要任务，目的是从在线产品评论中提取关键的方面术语。针对方面术语提取问题，提出基于注意力机制的双层BiReGU模型。该模型在传统BiLSTM模型的基础上，引入双嵌入机制和ReGU（Residual Gated Unit）作为辅助，以提高特征提取的能力。使用BiReGU学习文本特征表示，更好地捕捉词语间的长期依赖关系；在第一层BiReGU之后引入注意力机制，为文本中每个词语赋予不同的权重，得到融合特征后新的知识表示，再输入到第二层BiReGU中学习更加全局的文本特征表示，最后完成提取方面术语的任务。分别在SemEval 2014的Restaurant数据集和Laptop数据集做了相关的对比实验，实验结果证明了所提出方法的有效性。

关键词: 注意力机制, 方面术语提取, BiReGU, 方面级情感分析, 深度学习

ZHAO Lihua, WANG Chunli, CHU Yufeng. Attention-Based Double BiReGU Model for Aspect Term Extraction[J]. Computer Engineering and Applications, 2021, 57(22): 160-165.

赵丽华，王春立，初钰凤. 基于注意力双层BiReGU模型的方面术语提取方法[J]. 计算机工程与应用, 2021, 57(22): 160-165.

References

[1] BLOOM K，GARG N，ARGAMON S.Extracting appraisal expressions[C]//Annual Conference of the North American Chapter of the Association for Computational Linguistics，2007：308-315.
[2] ZHAO Y，QIN B，HU S，et al.Generalizing syntactic structures for product attribute candidate extraction[C]//Human Language Technologies：Conference of the North American Chapter of the Association of Computational Linguistics，Jun 2-4，2010，Los Angeles，2010：377-380.
[3] QIU G，LIU B，BU J，et al.Opinion word expansion and target extraction through double propagation[J].Computational Linguistics，2011，37（1）：9-27.
[4] YIN Y，WEI F，DONG L，et al.Unsupervised word and dependency path embeddings for aspect term extraction[C]//International Joint Conference on Artificial Intelligence，2016：2979-2985.
[5] XU H，LIU B，SHU L，et al.Double embeddings and CNN-based sequence labeling for aspect extraction[C]//56th Annual Meeting of the Association for Computational Linguistics，2018：592-598.
[6] MA D，LI S，WU F，et al.Exploring sequence-to-sequence learning in aspect term extraction[C]//57th Conference of the Association for Computational Linguistics，2019：3538-3547.
[7] HUANG Z，XU W，YU K.Bidirectional LSTM-CRF models for sequence tagging[J].arXiv：1508.01991，2015.
[8] MA X，HOVY E.End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF[C]//54th Annual Meeting of the Association for Computational Linguistics，2016：1064-1074.
[9] LI X，LAM W.Deep multi-task learning for aspect term extraction with memory interaction[C]//2017 Conference on Empirical Methods in Natural Language Processing，2017：2876-2882.
[10] 沈亚田，黄萱菁，曹均阔.使用深度长短时记忆模型对于评价词和评价对象的联合抽取[J].中文信息学报，2018，32（2）：110-119.
SHEN Y T，HUANG X J，CAO J K.Joint extraction of opinion targets and opinion words based on LSTM[J].Journal of Chinese Information Processing，2018，32（2）：110-119.
[11] WU C，WU F，WU S，et al.A hybrid unsupervised method for aspect term and opinion target extraction[J].Knowledge-Based Systems，2018，148：66-73.
[12] LUO H，LI T，LIU B，et al.Improving aspect term extraction with bidirectional dependency tree representation[J].IEEE/ACM Transactions on Audio，Speech and Language Processing，2019，27（7）：1201-1212.
[13] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems，2017：5998-6008.
[14] LUO H，LI T，LIU B，et al.DOER：dual cross-shared RNN for aspect term-polarity co-extraction[C]//57th Conference of the Association for Computational Linguistics，2019：591-601.
[15] HOCHREITER S，SCHMIDHUBER J.Long short-term memory[J].Neural Computation，1997，9（8）：1735-1780.
[16] GERS F A，SCHMIDHUBER J，CUMMINS F.Learning to forget：continual prediction with LSTM[J].Neural Computation，2000，12（10）：2451-2471.
[17] CHERNYSHEVICH M.IHS R&D Belarus：cross-domain extraction of product features using CRF[C]//8th International Workshop on Semantic Evaluation，2014：309-313.