Research on Multi-Dimensional End-to-End Phrase Recognition Algorithm Based on Background Knowledge

doi:10.3778/j.issn.1002-8331.2009-0422

Abstract

Abstract: At present, the deep end-to-end method based on supervised learning is mainly used in entity recognition and dependency analysis. There are two problems in this method：firstly, background knowledge cannot be introduced; secondly, multi-granularity and nested features of natural language cannot be recognized. In order to solve the above problems, this paper proposes a dependency syntax annotation rule based on phrase window, labels the Chinese phrase window data set（CPWD）, and designs a supporting multi-dimensional end-to-end phrase recognition model（MDM model）. The rule takes phrase as the minimum unit, divides sentences into seven nested phrase types, and indicates the dependency between phrases. MDM model can not only introduce background knowledge, recognize various nested phrases in sentences, but also recognize the dependency between phrases. The experimental results show that the annotation rule is easy to use and has no ambiguity. At the same time, the MDM model can deal with the problem of phrase nesting more effectively than the traditional end-to-end algorithm. The experiment on CPWD dataset shows that the MDM model can improve the [F1] value by more than 1 percentage point compared with the end-to-end method. The corresponding method is applied to the Chinese Metaphorical Emotion Analysis Competition of CCL2018, which improves by more than 1 percentage point and wins the first place.

Key words: natural language processing, annotation system, phrase recognition, dependency analysis

摘要： 目前，实体识别与依存关系分析，采用的主要是基于监督学习的深度端到端方法。这种方法存在两个问题：不能引入背景知识；不能识别出自然语言的多粒度、嵌套特征。为了解决以上问题，提出了基于短语窗口的依存句法标注规则，并标注了中文短语窗口数据集（CPWD），同时设计了配套的多维端到端短语识别模型（MDM模型）。该标注规则以短语为最小单位，把句子分成7类可嵌套的短语类型，同时标示出短语之间的依存关系。MDM模型不仅可以引入背景知识，识别出句子中的各类嵌套短语，而且可以识别出短语之间的依存关系。实验结果表明，该标注规则方便易用。同时，MDM模型比传统端到端算法能更有效地处理短语嵌套的问题。在CPWD数据集上实验，MDM模型比端到端方法在[F1]值上提高1个百分点以上。相应的方法应用到了CCL2018的中文隐喻情感分析比赛中，在原有基础上提升了1个百分点以上，并取得第一名成绩。

关键词: 自然语言处理, 标注体系, 短语识别, 依存分析

LIU Guang, TU Gang, LI Zheng, LIU Yijian, ZHAN Zhiqiang. Research on Multi-Dimensional End-to-End Phrase Recognition Algorithm Based on Background Knowledge[J]. Computer Engineering and Applications, 2022, 58(8): 147-155.

刘广, 涂刚, 李政, 刘译键, 占志强. 支持背景知识的多维端到端短语识别算法研究[J]. 计算机工程与应用, 2022, 58(8): 147-155.

References

[1] ABNEY S P.Parsing by chunks[M]//Principle-based parsing.Netherlands：Springer，1991：257-278.
[2] KUDO T，MATSUMOTO Y.Chunking with support vector machines[C]//Second Meeting of the North American Chapter of the Association for Computational Linguistics，2001.
[3] SHEN H，SARKAR A.Voting between multiple data representations for text chunking[C]//Advances in Artificial Intelligence.Berlin，Heidelberg：Springer，2005.
[4] MANCEV D.A sequential dual method for the structured ramp loss minimization[J].Facta Universitatis，2015，30（1）：13-27.
[5] 周强.基于规则的汉语基本块自动分析器[C]//第七届中文信息处理国际会议论文集（ICCC-2007），武汉，2007：137-142.
ZHOU Qiang.Rule-based automatic analyzer of Chinese basic blocks[C]//Proceedings of the 7th International Conference on Chinese Information Processing（ICCC-2007），Wuhan，2007：137-142.
[6] 周强.汉语基本块规则的自动学习和扩展进化[J].清华大学学报（自然科学版），2008，48（1）：88-91.
ZHOU Qiang.Automatic learning and refinement algorithm for Chinese base chunk rules[J].Journal of Tsinghua University（Science and Technology），2008，48（1）：88-91.
[7] 李超，孙健，关毅，等.基于最大熵模型的汉语基本块分析技术研巧[R].CIPS-ParsEval，2009.
LI Chao，SUN Jian，GUAN Yi，et al.Research on Chinese basic block analysis technology based on the maximum standby model[R].CIPS-ParsEval，2009.
[8] CHIU J P C，Nichols E.Named entity recognition with bidirectional LSTM-CNNs[J].Transactions of the Association for Computational Linguistics，2016，4：357-370.
[9] KURU O，CAN O A，YURET D.CharNER：character-level named entity recognition[C]//26th International Conference on Computational Linguistics，2016：911-921.
[10] 侯潇琪，王瑞波，李济洪.基于词的分布式实值表示的汉语基本块识别[J].中北大学学报（自然科学版），2013，34（5）：582-585.
HOU Xiaoqi，WANG Ruibo，LI Jihong.Identification of Chinese base chunk based on real-valued word distributed representations[J].Journal of North University of China（Natural Science Edition），2013，34（5）：582-585.
[11] 李国臣，党帅兵，王瑞波，等.基于字的分布表征的汉语基本块识别[J].中文信息学报，2014，28（6）：18-25.
LI Guochen，DANG Shuaibing，WANG Ruibo，et al.Chinese base-chunk identification based on distributed character representation[J].Journal of Chinese Information Processing，2014，28（6）：18-25.
[12] 徐菁.面向中文知识图谱的开放式文本信息抽取关键技术研究[D].北京：国防科技大学，2018.
XU Jing.Research on key technologies of open text information extraction for Chinese knowledge graph[D].Beijing：National University of Defense Technology，2018.
[13] 程钟慧，陈珂，陈刚，等.基于强化学习协同训练的命名实体识别方法[J].软件工程，2020，23（1）：7-11.
CHENG Zhonghui，CHEN Ke，CHEN Gang，et al.Named entity recognition method based on co-training of reinforcement learning[J].Software Engineer，2020，23（1）：7-11.
[14] 徐烈炯，沈阳.题元理论与汉语配价问题[J].代语言学，1998（3）：1-21.
XU Liejiong，SHEN Yang.Thematic theory and argument structure in Mandarin Chinese[J].Contemporary Linguistics，1998（3）：1-21.
[15] 刘宇红.生成语法中词汇语义与句法的界面研究[J].外语学刊，2011（5）：56-60.
LIU Yuhong.Lexical meaning and syntactical interface in the perspective of generative grammar[J].Foreign Language Research，2011（5）：56-60.
[16] 孙道功.基于大规模语义知识库的“词汇—句法语义”接口研究[J].语言文字应用，2016（2）：125-134.
SUN Daogong.On the interface of “lexicon-syntactic semantics” based on a large-scale semantic knowledge base[J].Applied Linguistics，2016（2）：125-134.
[17] 亢世勇，许小星，马永腾.施事、受事句法实现的义类制约[J].语文研究，2011（4）：36-40.
KANG Shiyong，XU Xiaoxing，MA Yongteng.Semantic restriction in the realization of acting and receiving syntax[J].Linguistic Research，2011（4）：36-40.
[18] MCDONALD R，LERMAN K，PEREIRA F.Multilingual dependency analysis with a two-stage discriminative parser[C]//Tenth Confrence on Computational Natural Language Learning，2006：216-220.
[19] NIVRE J，HALL J，NILSSON J，et al.Labeled pseudo-projective dependency parsing with support vector machines[C]//Tenth Conference on Computational Natural Language Learning，2006.
[20] REN H，JI D，WAN J，et al.Parsing syntactic and semantic dependencies for multiple languages with a pipeline approach[C]//Proceedings of the Thirteenth Conference on Computational Natural Language Learning（CoNLL 2009）：Shared Task，2009：97-102.
[21] CHE W，LI Z，LI Y，et al.Multilingual dependency-based syntactic and semantic parsing[C]//Thirteenth Conference on Computational Natural Language Learning，2009：49-54.
[22] DYER C，BALLESTEROS M，LING W，et al.Transition-based dependency parsing with stack long short-term memory[J].arXiv：1505.08075，2015.
[23] JI T，WU Y，LAN M.Graph-based dependency parsing with graph neural networks[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：2475-2485.
[24] WANG Y，CHE W，GUO J，et al.A neural transition-based approach for semantic dependency graph parsing[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2018.
[25] FRIED D，KLEIN D.Policy gradient as a proxy for dynamic oracles in constituency parsing[C]//Proceedings of the ACL，2018：469-476.
[26] 丁伟伟，常宝宝.基于语义组块分析的汉语语义角色标注[J].中文信息学报，2009，23（5）：53-61.
DING Weiwei，CHANG Baobao.Chinese semantic role labeling based on semantic chunking[J].Journal of Chinese Information Processing，2009，23（5）：53-61.
[27] 王丽杰.汉语语义依存分析研究[D].哈尔滨：哈尔滨工业大学，2010.
WANG Lijie.Analysis of Chinese semantic dependence[D].Harbin：Harbin Institute of Technology，2010.
[28] 王倩，罗森林，韩磊，等.基于谓词及句义类型块的汉语句义类型识别[J].中文信息学报，2014，28（2）：8-16.
WANG Qian，LUO Senlin，HAN Lei，et al.Chinese sentential semantic type recognition based on predicate and sentential semantic type chunk[J].Journal of Chinese Information Processing，2014，28（2）：8-16.
[29] SCHMIDHUBER J，HOCHREITER S.Long short-term memory[J].Neural Computation，1997，9（8）：1735-1780.
[30] DEVLIN J，CHANG M W，LEE K，et al.Bert：pre-training of deep bidirectional transformers for language understanding[J].arXiv：1810.04805，2018.
[31] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems，2017：5998-6008.
[32] HUANG Z，XU W，YU K.Bidirectional LSTM-CRF models for sequence tagging[J].arXiv：1508.01991，2015.
[33] ZENG Y，YANG H，FENG Y，et al.A convolution BiLSTM neural network model for Chinese event extraction[C]//Natural Language Understanding and Intelligent Applications.Cham：Springer，2016：275-287.