Fin-BERT-Based Event Extraction Method for Chinese Financial Domain

doi:10.3778/j.issn.1002-8331.2304-0224

Abstract

Abstract: Event extraction aims to extract human-interest information from massive amounts of unstructured text. Currently, most existing event extraction methods are based on general corpora and rarely consider domain-specific prior knowledge. Moreover, most methods cannot handle well the case where multiple events exist in the same document, and they perform poorly when faced with a large number of negative examples. To address these issues, this paper proposes a model called Fin-PTPCG based on Fin-BERT (financial bidirectional encoder representation from Transformers) and PTPCG (pseudo-trigger-aware pruned complete graph). This method fully utilizes the expression ability of the Fin-BERT pre-training model and incorporates domain-specific prior knowledge during the encoding stage. In the event detection module, multiple binary classifiers are stacked to ensure that the model can effectively identify the situation of multiple events in a document and screen out negative examples. Combined with the decoding module of the PTPCG model, entities are extracted and connected into a complete graph and pruned by calculating a similarity matrix. The problem of unlabeled triggers is solved by selecting pseudo-triggers. Finally, the event extraction is achieved by the event classifier. This method achieves a 0.7 and 3.7?percentage points improvement in F1 score compared to the baselines on the ChFinAnn and Duee-fin datasets for the event extraction task.

Key words: event extraction, event detection, information extraction, natural language processing

摘要： 事件抽取旨在从海量的非结构化的事件相关文本中抽取出人类感兴趣的内容，目前现有的事件抽取方法大多数基于通用语料，很少考虑到领域内的先验知识，并且现有的方法大多数不能很好地处理同一文档包含多个事件的情况，面对存在较多负面样例的测试也表现不佳。针对上述问题提出了一种基于Fin-BERT（financial bidirectional encoder representation from Transformers）和PTPCG（pseudo-trigger-aware pruned complete graph）的模型Fin-PTPCG，该方法充分利用Fin-BERT预训练模型的表达能力，在编码阶段融入领域内的先验知识，并且在事件检测模块采用多个二元分类器叠加的方式，保证模型可以有效识别一篇文档内存在多事件的情况并筛除掉负面样例，抽取实体之后将实体连接成完全图并通过计算相似度矩阵进行剪枝，通过选择伪触发器解决无标注触发词的问题，最后接入事件分类器实现事件抽取。该方法在ChFinAnn和Duee-fin数据集上事件抽取任务的F1值相比于基线方法分别取得了0.7个百分点和3.7个百分点的提升。

关键词: 事件抽取, 事件检测, 信息抽取, 自然语言处理

LI Yi, GENG Chaoyang, YANG Dan. Fin-BERT-Based Event Extraction Method for Chinese Financial Domain[J]. Computer Engineering and Applications, 2024, 60(14): 123-132.

李熠, 耿朝阳, 杨丹. 基于Fin-BERT的中文金融领域事件抽取方法[J]. 计算机工程与应用, 2024, 60(14): 123-132.

References

[1] 郭喜跃, 何婷婷. 信息抽取研究综述[J]. 计算机科学, 2015, 42(2): 14-17.
GUO X Y, HE T T. Survey about research on information extraction[J]. Computer Science, 2015, 42(2): 14-17.
[2] BOSSELUT A, LE BRAS R, CHOI Y. Dynamic neuro-symbolic knowledge graph construction for zero-shot commonsense question answering[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(6): 4923-4931.
[3] LI M. ZHU Y, WANG R. An empirical study on utilizing neural network for event information retrieval[C]//Proceedings of the 2020 International Conference on Computer Science and Communication Technology, 2020: 51-56.
[4] LI M, ZAREIAN A, LIN Y, et al. GAIA: a fine-grained multimedia knowledge extraction system[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2020: 77-86.
[5] 胡瑞娟, 周会娟, 刘海砚, 等. 基于深度学习的篇章级事件抽取研究综述[J]. 计算机工程与应用, 2022, 58(24): 47-60.
HU R J, ZHOU H J, LIU H Y, et al. Survey on document-level event extraction based on deep learning[J]. Computer Engineering and Applications, 2022, 58(24): 47-60.
[6] RILOFF E. Automatically constructing a dictionary for information extraction tasks[C]//Proceedings of the 11th National Conference on Artificial Intelligence, 1993: 811-816.
[7] LI Q, JI H, HUANG L. Joint event extraction via structured prediction with global features[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, 2013: 73-82.
[8] 陈星月, 倪丽萍, 倪志伟. 基于ELECTRA模型与词性特征的金融事件抽取方法研究[J]. 数据分析与知识发现, 2021, 5(7): 36-47.
CHEN X Y, NI L P, NI Z W. Extracting financial events with ELECTRA and part-of-speech[J]. Data Analysis and Knowledge Discovery, 2021, 5(7): 36-47.
[9] 万齐智, 万常选, 胡蓉, 等. 基于句法语义依存分析的中文金融事件抽取[J]. 计算机学报, 2021, 44(3): 508-530.
WAN Q Z, WAN C X, HU R, et al. Chinese financial event extraction base on syntactic and semantic dependency parsing[J]. Chinese Journal of Computers, 2021, 44(3): 508-530.
[10] WANG P, DENG Z, CUI R. TDJEE: a document-level joint model for financial event extraction[J]. Electronics, 2021, 10(7): 824.
[11] NGUYEN T H, GRISHMAN R. Event detection and domain adaptation with Convolutional neural networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 2015: 365-371.
[12] NGUYEN T H, CHO K, GRISHMAN R. Joint event extraction via recurrent neural networks[C]//Proceedings of the Annual Conference on North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016: 300-309.
[13] WADDEN D, WENNBERG U, LUAN Y, et al. Entity, relation, and event extraction with contextualized span representations[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, 2019: 5784-5789.
[14] DU X, CARDIE C. Document-level event role filler extraction using multi-granularity contextualized encoding[C]//Proceedings of the Association for Computational Linguistics, 2020: 634-644.
[15] YANG H, CHEN Y, LIU K, et al. DCFEE: a document-level Chinese financial event extraction system based on automatically labeled training data[C]//Proceedings of the Association for Computational Linguistics, 2018: 50-55.
[16] ZHENG S, CAO W, XU W, et al. Doc2EDAG: an end-to-end document-level framework for Chinese financial event extraction[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, 2019: 337-346.
[17] XU R, LIU T, LI L, et al. Document-level event extraction via heterogeneous graph-based interaction model with a tracker[C]//Proceedings of the Joint Conference on 59th Annual Meeting of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, 2021: 3533-3546.
[18] ZHU T, QU X, CHEN W L, et al. Efficient document-level event extraction via pseudo-trigger-aware pruned complete graph[C]//Proceedings of International Joint Conference on Artificial Intelligence, 2022: 4552-4558.
[19] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference on North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[20] LI X. DuEE-Fin: a document-level event extraction dataset in the financial domain released by baidu[EB/OL]. (2021)[2023-04-06]. https://aistudio.baidu.com/aistudio/competition/detail/46.
[21] HERMANS J R, SPANAKIS G, M?CKEL R. Accumulated gradient normalization[C]//Proceedings of the Asian Conference on Machine Learning, 2017: 439-454.
[22] LIU Y, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized BERT pretraining approach[J]. arXiv:1907.11692, 2019.