Event Temporal Relation Identification with Dependency Information

doi:10.3778/j.issn.1002-8331.2110-0338

Abstract

Abstract: Event temporal relation identification is an important natural language understanding task. This task helps readers to analyze the content of the article and clarify the development of events. Most of existing event temporal relation identification methods focus on extracting the semantic information around the trigger, but the information in the event sentence is relatively scattered, which leads the model to lose part of the event information. To address the above problems, a bidirectional dependency attention mechanism is proposed to aggregate event information. A bidirectional dependency matrix is constructed based on the parent-child node information of words, and syntactic information is integrated into word embeddings. Combining this mechanism with Bi-LSTM（bidirectional long short term memory） network, this model can significantly improve the performance of event temporal relation identification. It conducts comparative experiments on Vietnamese datasets and English datasets, the results show that this method is superior to the mainstream neural network methods.

Key words: event temporal relation, self-attention mechanism, event relation identification, adversarial training, dependency parsing

摘要： 事件时序关系识别有助于读者理清文章脉络，把握全局发展趋势，是重要的自然语言理解任务之一。现有的事件时序关系识别方法专注于提取事件触发词前后的局部信息，然而事件句中的事件信息分布较为分散，导致模型在编码过程中丢失部分事件信息。针对上述问题，针对文本特征提出一种双路依存注意力机制来聚合事件句信息，通过单词的父子节点信息构建出双路依存矩阵，将句法信息融入到词嵌入中。将该机制与双向长短期记忆网络（bidirectional long short term memory，Bi-LSTM）结合，可以使事件时序关系模型的性能得到显著提高。该文在越南语数据集与英语数据集上进行对比实验，结果表明所提方法优于主流的神经网络方法。

关键词: 事件时序关系, 注意力机制, 事件关系识别, 对抗训练, 依存句法

LI Liangyi, ZHANG Yafei, GUO Junjun, GAO Shengxiang, YU Zhengtao. Event Temporal Relation Identification with Dependency Information[J]. Computer Engineering and Applications, 2023, 59(7): 110-117.

李良毅, 张亚飞, 郭军军, 高盛祥, 余正涛. 融入依存句法信息的事件时序关系识别[J]. 计算机工程与应用, 2023, 59(7): 110-117.

References

[1] NING Q，FENG Z，ROTH D.A structured learning approach to temporal relation extraction[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing，2017：1027-1037.
[2] HAN R，REN X，PENG N.Deer：a data efficient language model for event temporal reasoning[J].arXiv：2012. 15283，2020.
[3] BUGLIARELLO E，OKAZAKI N.Enhancing machine translation with dependency-aware self-attention[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics，2020：1618-1627.
[4] VERHAGEN M，PUSTEJOVSKY J.Temporal processing with the TARSQI toolkit[C]//Proceedings of COLING 2008，2008：189-192.
[5] BETHARD S.Cleartk-timeml：a minimalist approach to tempeval 2013[C]//Proceedings of 2nd Joint Conference on Lexical and Computational Semantics（*SEM），Proceedings of the 7th International Workshop on Semantic Evaluation，2013：10-14.
[6] SAURí R，LITTMAN J，KNIPPEN B，et al.TimeML annotation guidelines[J].Version，2006，1（1）：31.
[7] PUSTEJOVSKY J，HANKS P，SAURI R，et al.The timebank corpus[C]//Proceedings of Corpus Linguistics，2003：647-656.
[8] CHAMBERS N，WANG S，JURAFSKY D.Classifying temporal relations between events[C]//Proceedings of the 45th Annual Meeting of ACL，2007：173-176.
[9] MANI I，VERHAGEN M，WELLNER B，et al.Machine learning of temporal relations[C]//Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the ACL，2006：753-760.
[10] CHAMBERS N，CASSIDY T，MCDOWELL B，et al.Dense event ordering with a multi-pass architecture[J].Transactions of the Association for Computational Linguistics，2014，2（11）：273-284
[11] VASHISHTHA S，VAN DURME B，WHITE A.Fine-grained temporal relation extraction[C]//Proceedings of the 57th Annual Meeting of ACL，2019：2906-2919.
[12] NING Q，FENG Z，WU H，et al.Joint reasoning for temporal and causal relations[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics，2018：2278-2288.
[13] MENG Y，RUMSHISKY A.Context-aware neural model for temporal information extraction[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics，2018.
[14] HAN R，NING Q，PENG N.Joint event and temporal relation extraction with shared representations and structured prediction[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing，2019：434-444.
[15] HAN R，ZHOU Y，PENG N.Domain knowledge empowered structured neural net for end-to-end event temporal relation extraction[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing，2020：5717-5729.
[16] CHENG F，MIYAO Y.Classifying temporal relations by bidirectional LSTM over dependency paths[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics，2017：1-6.
[17] MIYATO T，DAI A M，GOODFELLOW I.Adversarial training methods for semi-supervised text classification[C]//Proceedings of International Conference on Learning Representations，2017：1050.
[18] LIN T Y，GOYAL P，GIRSHICK R，et al.Focal loss for denseobject detection[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2980-2988.
[19] CASSIDY T，MCDOWELL B，CHAMBERS N，et al.An annotation framework for dense event ordering[C]//Proceedings of the 52nd Annual Meeting of ACL，2014：501-506.
[20] NING Q，WU H，ROTH D.A multi-axis annotation scheme for event temporal relations[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics，2018：1318-1328.
[21] ZHOU Y，YAN Y，HAN R，et al.Clinical temporal relation extraction with probabilistic soft logic regularization and global inference[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2021.
[22] WANG H，CHEN M，ZHANG H，et al.Joint Constrained learning for event-event relation extraction[C]//Proceedings of Conference on Empirical Methods in Natural Language Processing，2020：696.
[23] MA M D，SUN J，YANG M，et al.EventPlus：a temporal event understanding pipeline[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies：Demonstrations，2021：56-65.