Causal Relationship Representation Enhanced Cross-Domain Named Entity Recognition

doi:10.3778/j.issn.1002-8331.2311-0068

Abstract

Abstract: Cross-domain named entity recognition is of great value in real-world applications, especially in few-shot scenarios with scarce data in the target domain. However, the existing methods are mainly cross-domain entity capability migration achieved by feature representation or model parameter sharing, which does not fully consider the false correlation problem due to sample selection bias. To address the problem of false correlation in cross-domain, a causal representation-enhanced cross-domain named entity recognition model is proposed, in which the semantic feature representation of the source domain is first fused with that of the target domain to generate an enhanced contextual semantic feature representation. Then, the causal relationships between the enhanced feature variables and the labels are captured by a structural causal model. Finally, causal intervention and counterfactual inference strategies are applied in the target domain to extract the presence of direct causal effects, thus further mitigating the problem of spurious correlation between features and labels. The method is experimented on a public dataset and the experimental results are significantly improved.

Key words: cross-domain named entity recognition, transfer learning, causal relationship, structural causal model, semantic feature representation

摘要： 跨领域命名实体识别在现实应用中，尤其在目标领域数据稀缺的小样本场景中具有重要价值。然而，现有方法主要是通过特征表示或模型参数共享实现的跨领域实体能力迁移，未充分考虑由于样本选择偏差而引起的虚假相关性问题。为了解决跨领域中的虚假相关性问题，提出一种因果关系表示增强的跨领域命名实体识别模型，将源域的语义特征表示与目标域的语义特征表示进行融合，生成一种增强的上下文语义特征表示。通过结构因果模型捕捉增强后的特征变量与标签之间的因果关系。在目标域中应用因果干预和反事实推断策略，提取存在的直接因果效应，从而进一步缓解特征与标签之间的虚假相关性问题。该方法在公共数据集上进行了实验，实验结果得到了显著提高。

关键词: 跨领域命名实体识别, 迁移学习, 因果关系, 结构因果模型, 语义特征表示

LIU Xiaoming, CAO Mengyuan, YANG Guan, LIU Jie, WANG Hang. Causal Relationship Representation Enhanced Cross-Domain Named Entity Recognition[J]. Computer Engineering and Applications, 2024, 60(18): 176-188.

刘小明, 曹梦远, 杨关, 刘杰, 王杭. 因果关系表示增强的跨领域命名实体识别[J]. 计算机工程与应用, 2024, 60(18): 176-188.

References

[1] YADAV V, BETHARD S. A survey on recent advances in named entity recognition from deep learning models[C]//Proceedings of the 27th International Conference on Computational Linguistics, 2018: 2145-2158.
[2] 何玉洁, 杜方, 史英杰, 等. 基于深度学习的命名实体识别研究综述[J]. 计算机工程与应用, 2021, 57(11): 21-36.
HE Y J, DU F, SHI Y J, et al. Survey of named entity recognition based on deep learning[J]. Computer Engineering and Applications, 2021, 57(11): 21-36.
[3] LIU Z, XU Y, YU T, et al. CrossNER: evaluating cross-domain named entity recognition[C]//Proceedings of the 21st AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2021: 13452-13460.
[4] 张文韩, 刘小明, 杨关, 等.多层结构化语义知识增强的跨领域命名实体识别[J].计算机研究与发展, 2023, 60(12):2854-2876.
ZHANG W H, LIU X M, YANG G, et al. Cross-domain named entity recognition of multi-level structured semantic knowledge enhancement[J].Journal of Computer Research and Development, 2023, 60(12): 2854-2876.
[5] JIA C, ZHANG Y. Multi-cell compositional LSTM for NER domain adaptation[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2020: 5906-5917.
[6] LIU Z, WINATA G I, XU P, et al. Coach: a coarse-to-fine approach for cross-domain slot filling[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2020: 19-25.
[7] HU J P, ZHAO H, GUO D D, et al. A label-aware autoregressive framework for cross-domain NER[C]//Proceedings of Findings of the Association for Computational Linguistics(NAACL 2022), 2022: 2222-2232.
[8] LEE J Y, DERNONCOURT F, SZOLOVITS P. Transfer learning for named-entity recognition with neural networks[J]. arXiv:1705.06273, 2017.
[9] ZHENG J, CHEN H, MA Q. Cross-domain named entity recognition via graph matching[C]//Proceedings of Findings of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2022: 2670-2680.
[10] CHEN X, LI L, FEI Q, et al. One model for all domains: collaborative domain-prefix tuning for cross-domain NER[J]. arXiv:2301.10410, 2023.
[11] FEDER A, KEITH K A, MANZOOR E, et al. Causal inference in natural language processing: estimation, prediction, interpretation and beyond[J]. Transactions of the Association for Computational Linguistics, 2022, 10: 1138-1158.
[12] NAN G, ZENG J, QIAO R, et al. Uncovering main causalities for long-tailed information extraction[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021: 9683-9695.
[13] ZENG X, LI Y, ZHAI Y, et al. Counterfactual generator: a weakly-supervised method for named entity recognition[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020: 7270-7280.
[14] NAN J, WANG Y, WANG C. Rating prediction model based on causal inference debiasing method in recommendation[J]. Chinese Journal of Electronics, 2023, 32(4): 932-940.
[15] KAUSHIK D, HOVY E, LIPTON Z. Learning the difference that makes a difference with counterfactually-augmented data[C]//Proceedings of the International Conference on Learning Representations, 2019.
[16] KAUSHIK D, SETLUR A, HOVY E H, et al. Explaining the efficacy of counterfactually augmented data[C]//Proceedings of the International Conference on Learning Representations, 2020.
[17] WANG Z, CULOTTA A. Identifying spurious correlations for robust text classification[C]//Proceedings of Findings of the Association for Computational Linguistics (EMNLP 2020), 2020: 3431-3440.
[18] XU N, WANG F, LI B, et al. Does your model classify entities reasonably? Diagnosing and mitigating spurious correlations in entity typing[C]//Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022: 8642-8658.
[19] MOGHIMIFAR F, HAFFARI G, BAKTASHMOTLAGH M. Domain adaptative causality encoder[C]//Proceedings of the The 18th Annual Workshop of the Australasian Language Technology Association, 2020: 1-10.
[20] 李家宁, 熊睿彬, 兰艳艳, 等. 因果机器学习的前沿进展综述[J].?计算机研究与发展, 2023, 60(1): 59-84.
LI J N, XIONG R B, LAN Y Y et, al. Overview of the frontier progress of causal machine learning[J].Journal of Computer Research and Development, 2023, 60(1): 59-84.
[21] PEARL J, GLYMOUR M, JEWELL N P. Causal inference in statistics: a primer[M]. [S.l.]: John Wiley & Sons, 2016.
[22] FRITZLER A, LOGACHEVA V, KRETOV M. Few-shot classification in named entity recognition task[C]//Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, 2019: 993-1000.
[23] TONG M, WANG S, XU B, et al. Learning from miscellaneous other-class words for few-shot named entity recognition[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021: 6236-6247.
[24] CUI L, WU Y, LIU J, et al. Template-based named entity recognition using BART[C]//Proceedings of Findings of the Association for Computational Linguistics (ACL-IJCNLP 2021), 2021: 1835-1845.
[25] MA R, ZHOU X, GUI T, et al. Template-free prompt tuning for few-shot NER[C]//Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2022: 5721-5732.
[26] GU S, FENG Y, LIU Q. Improving domain adaptation translation with domain invariant and specific information[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long and Short Papers), 2019: 3081-3091.
[27] CHEN J, LIN H, HAN X, et al. Honey or poison? Solving the trigger curse in few-shot event detection via causal intervention[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021: 8078-8088.
[28] TANG K, HUANG J, ZHANG H. Long-tailed classification by keeping the good and removing the bad momentum causal effect[C]//Advances in Neural Information Processing Systems, 2020: 1513-1524.
[29] ZHANG W, LIN H, HAN X, et al. De-biasing distantly supervised named entity recognition via causal intervention[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021: 4803-4813.
[30] PEYRé G, CUTURI M, SOLOMON J. Gromov-wasserstein averaging of kernel and distance matrices[C]//Proceedings of the 33rd International Conference on International Conference on Machine Learning, 2016: 2664-2672.
[31] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[C]//Proceedings of International Conference on Learning Representations, 2016.
[32] KENTON J D M W C, TOUTANOVA L K. BERT: pre-training of deep bidirectional apr transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long and Short Papers), 2019: 4171-4186
[33] JIE Z, LU W. Dependency-guided LSTM-CRF for named entity recognition[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019: 3862-3872.
[34] LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016: 260-270.
[35] SANG E F T K, DE MEULDER F. Introduction to the CoNLL-2003 shared task: language-independent named entity recognition[C]//Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL 2003. Stroudsburg, PA: ACL, 2003: 142-147.
[36] NéDELLEC C, BOSSY R, KIM J, et al. Overview of BioNLP shared task 2013[C]//Proceedings of the BioNLP shared task 2013 Workshop. Stroudsburg, PA: ACL, 2013: 1-7.
[37] LIU J, PASUPAT P, CYPHERS S, et al. Asgard: a portable archi tecture for multilingual dialogue systems[C]//Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 2013: 8386-8390.
[38] LIU J, PASUPAT P, WANG Y, et al. Query understanding enhanced by hierarchical parsing structures[C]//Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, 2013: 72-77.
[39] PASZKE A, GROSS S, MASSA F, et al. PyTorch: an imperative style, high-performance deep learning library[C]//Advances in Neural Information Processing Systems, 2019:8024-8035.
[40] LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. PA: ACL, 2016: 260-270.