Review of Data-Driven Approaches to Chinese Named Entity Recognition

doi:10.3778/j.issn.1002-8331.2312-0260

Abstract

Abstract: Chinese named entity recognition (CNER) is a key step in Chinese information extraction task, which is the basis of downstream tasks such as question answering system, machine translation and knowledge mapping, and its methods are mainly categorized into two main types: knowledge-driven and data-driven. However, the traditional knowledge-driven methods based on rules, dictionaries and machine learning have the problems of ignoring contextual semantic information, high computational cost and low recall rate, which limit the development of CNER technology. Firstly, the definition and development history of CNER are introduced. Secondly, the typical datasets, training tools, sequence annotation methods and model evaluation indexes for CNER tasks are organized in detail. Thirdly, the data-driven methods are summarized and divided into methods based on deep learning, pre-trained language models and joint extraction of Chinese entity relations, and the practical application scenarios of data-driven methods in different fields are analyzed. Finally, the future research direction of CNER task is outlooked to provide some reference for the proposal of new methods.

Key words: Chinese named entity recognition, data-driven, deep learning, knowledge graph

摘要： 中文实体抽取（Chinese named entity recognition，CNER）是中文信息抽取任务中的关键一步，是问答系统、机器翻译和知识图谱等下游任务的基础，其方法主要分为知识驱动和数据驱动两大类。然而基于规则、词典与机器学习的传统知识驱动方法存在忽视上下文语义信息、计算成本高和低召回率的问题，限制了CNER技术的发展。介绍了CNER的定义和发展历程。详细整理了CNER任务的典型数据集、训练工具、序列标注方式和模型评价指标。对基于数据驱动的方法进行了总结，将数据驱动的方法划分为基于深度学习、预训练语言模型和中文实体关系联合抽取等方法，并分析了数据驱动方法在不同领域的实际应用场景。对CNER任务的未来研究方向进行了展望，为新方法的提出提供一定参考。

关键词: 中文实体抽取, 数据驱动, 深度学习, 知识图谱

XIAO Lei, CHEN Zhenjia. Review of Data-Driven Approaches to Chinese Named Entity Recognition[J]. Computer Engineering and Applications, 2024, 60(16): 34-48.

肖蕾, 陈镇家. 数据驱动的中文实体抽取方法综述[J]. 计算机工程与应用, 2024, 60(16): 34-48.

References

[1] 刘浏, 王东波. 命名实体识别研究综述[J]. 情报学报, 2018, 37(3): 329-340.
LIU L, WANG D B. A review on named entity recognition[J]. Journal of the China Society for Scientific and Technical Information, 2018, 37(3): 329-340.
[2] RAU L F. Extracting company names from text[C]//Proceedings of the 7th IEEE Conference on Artificial Intelligence Application, 1991: 29-32.
[3] GRISHMAN R, SUNDHEIM B M. Message understanding conference-6: a brief history[C]//Proceedings the 16th International Conference on Computational Linguistics, 1996: 466-471.
[4] NADEAU D, SEKINE S. A survey of named entity recognition and classification[J]. Lingvisticae Investigationes, 2007, 30(1): 3-26.
[5] MOLLá D, VAN ZAANEN M, SMITH D. Named entity recognition for question answering[C]//Proceedings of the Australasian Language Technology Workshop 2006, 2006: 51-58.
[6] GUO J, XU G, CHENG X, et al. Named entity recognition in query[C]//Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2009: 267-274.
[7] BABYCH B, HARTLEY A. Improving machine translation quality with automatic named entity recognition[C]//Proceedings of the 7th International EAMT Workshop on MT and Other Language Technology Tools, Improving MT Through Other Language Technology Tools, Resource and Tools for Building MT at EACL 2003, 2003.
[8] ETZIONI O, CAFARELLA M, DOWNEY D, et al. Unsupervised named entity extraction from the web: an experimental study[J]. Artificial Intelligence, 2005, 165(1): 91-134.
[9] CHINCHOR N A. Overview of MUC-7/MET-2[R]. San Diego: Science Applications International Corp, 1998.
[10] SEKINE S, ISAHARA H. IREX: IR & IE evaluation project in Japanese[C]//Proceedings of the 2nd International Conference on Language Resources and Evaluation, 2000: 1977-1980.
[11] SANG T K, ERIK F. Introduction to the CoNLL-2002 shared task: language-independent named entity recognition[C]//Proceedings of the 6th Conference on Natural Language Learning, 2002: 1-4.
[12] SANG E F, DE MEULDER F. Introduction to the CoNLL-2003 shared task: language-independent named entity recognition[C]//Proceedings of the 7th Conference on Natural Language Learning, 2003: 142-147.
[13] DODDINGTON G R, MITCHELL A, PRZYBOCKI M A, et al. The automatic content extraction (ACE) program-tasks, data, and evaluation[C]//Proceedings of the 4th International Conference on Language Resources and Evaluation, 2004: 837-840.
[14] SANTOS D, SECO N, CARDOSO N, et al. HAREM: an advanced NER evaluation contest for portuguese[C]//Proceedings of the 5th International Conference on Language Resources and Evaluation, Genoa, 2006: 1986-1991.
[15] ZHU P, CHENG D, YANG F, et al. Improving Chinese named entity recognition by large-scale syntactic dependency graph[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2022, 30: 979-991.
[16] 康怡琳, 孙璐冰, 朱容波, 等. 深度学习中文命名实体识别研究综述[J]. 华中科技大学学报(自然科学版), 2022, 50(11): 44-53.
KANG Y L, SUN L B, ZHU R B, et al. Survey on Chinese named entity recognition with deep learning[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2022, 50(11): 44-53.
[17] LEVOW G A. The third international Chinese language processing bakeoff: word segmentation and named entity recognition[C]//Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing, 2006: 108-117.
[18] LI Z, SUN M. Punctuation as implicit annotations for Chinese word segmentation[J]. Computational Linguistics, 2009, 35(4): 505-512.
[19] ZHANG Y, YANG J. Chinese NER using lattice LSTM[J]. arXiv:1805.02023, 2018.
[20] PENG N, DREDZE M. Named entity recognition for Chinese social media with jointly trained embeddings[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015: 548-554.
[21] XU L, DONG Q, LIAO Y, et al. CLUENER2020: fine-grained named entity recognition dataset and benchmark for Chinese[J]. arXiv:2001.04351, 2020.
[22] REIMERS N, GUREVYCH I. Optimal hyperparameters for deep LSTM-Networks for sequence labeling tasks[J]. arXiv:1707.06799, 2017.
[23] PRADHAN S, MOSCHITTI A, XUE N, et al. CoNLL-2012 shared task: modeling multilingual unrestricted coreference in OntoNotes[C]//Proceedings of the 2012 Joint Conference on EMNLP and CoNLL-Shared Task, 2012: 1-40.
[24] LI J, SUN A, HAN J, et al. A survey on deep learning for named entity recognition[J]. IEEE Transactions on Knowledge and Data Engineering, 2020, 34(1): 50-70.
[25] MICHAEK C, YORAM S. Unsupervised models for named entity classification[C]//Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, 1999：100-110.
[26] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
[27] HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7): 1527-1554.
[28] RODRíGUEZ P, BAUTISTA M A, GONZALEZ J, et al. Beyond one-hot encoding: lower dimensional target embedding[J]. Image and Vision Computing, 2018, 75: 21-31.
[29] SONG F, LIU S, YANG J. A comparative study on text representation schemes in text categorization[J]. Pattern Analysis and Applications, 2005, 8: 199-209.
[30] ZHANG W, YOSHIDA T, TANG X. A comparative study of TF-IDF, LSI and multi-words for text classification[J]. Expert Systems with Applications, 2011, 38(3): 2758-2765.
[31] SHAHMIRZADI O, LUGOWSKI A, YOUNGE K. Text similarity in vector space models: a comparative study[C]//Proceedings of the 18th IEEE International Conference on Machine Learning and Applications, 2019: 659-666.
[32] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceddings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, 2013: 3111-3119.
[33] CAO S, LU W, ZHOU J, et al. CW2Vec: learning Chinese word embeddings with stroke n-gram information[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, 2018: 5053-5061.
[34] PENNINGTON J, SOCHER R, MANNING C D. GloVe: global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, 2014: 1532-1543.
[35] JOULIN A, GRAVE E, BOJANOWSKI P, et al. Bag of tricks for efficient text classification[J]. arXiv:1607.01759, 2016.
[36] SARZYNSKA-WAWER J, WAWER A, PAWLAK A, et al. Detecting formal thought disorder by deep contextualized word representations[J]. Psychiatry Research, 2021, 304: 114135.
[37] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems, 2017: 5998-6008.
[38] RADFORD A, WU J, CHILD R, et al. Language models are unsupervised multitask learners[J]. OpenAI Blog, 2019, 1(8): 9.
[39] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[40] LAN Z, CHEN M, GOODMAN S, et al. ALBERT: a lite BERT for self-supervised learning of language representations[J]. arXiv:1909.11942, 2019.
[41] CUI Y, CHE W, LIU T, et al. Pre-training with whole word masking for Chinese BERT[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 3504-3514.
[42] SUN Y, WANG S, LI Y, et al. ERNIE: enhanced representation through knowledge integration[J]. arXiv:1904.09223, 2019.
[43] YANG Z, DAI Z, YANG Y, et al. XLNet: generalized autoregressive pretraining for language understanding[J]. arXiv:1906.08237, 2019.
[44] COLLOBERT R, WESTON J. A unified architecture for natural language processing: deep neural networks with multitask learning[C]//Proceedings of the 25th International Conference on Machine Learning, 2008: 160-167.
[45] SHEN Y, YUN H, LIPTON Z C, et al. Deep active learning for named entity recognition[J]. arXiv:1707.05928, 2017.
[46] ZHENG S, WANG F, BAO H, et al. Joint extraction of entities and relations based on a novel tagging scheme[J]. arXiv:1706.05075, 2017.
[47] NGUYEN T H, SIL A, DINU G, et al. Toward mention detection robustness with recurrent neural networks[J]. arXiv:1602.07749, 2016.
[48] CHEN X, QIU X, ZHU C, et al. Long short-term memory neural networks for Chinese word segmentation[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015: 1197-1206.
[49] DONG C, ZHANG J, ZONG C, et al. Character-based LSTM-CRF with radical-level features for Chinese named entity recognition[C]//Proceedings of the 5th CCF Conference on Natural Language Processing and Chinese Computing, and the 24th International Conference on Computer Processing of Oriental Languages, Kunming, 2016: 239-250.
[50] YANG J, LIANG S, ZHANG Y. Design challenges and misconceptions in neural sequence labeling[J]. arXiv:1806. 04470, 2018.
[51] MA X, HOVY E. End-to-end sequence labeling via bidirectional LSTM-CNNS-CRF[J]. arXiv:1603.01354, 2016.
[52] ZHANG X, ZHAO J, LECUN Y. Character-level convolutional networks for text classification[C]//Advances in Neural Information Processing Systems 28, 2015: 649-657.
[53] MIKOLOV T, KARAFIáT M, BURGET L, et al. Recurrent neural network based language model[J]. Interspeech, 2010, 2(3): 1045-1048.
[54] MA R, PENG M, ZHANG Q, et al. Simplify the usage of lexicon in Chinese NER[J]. arXiv:1908.05969, 2019.
[55] LI X, YAN H, QIU X, et al. FLAT: Chinese NER using flat-lattice transformer[J]. arXiv:2004.11795, 2020.
[56] WU S, SONG X, FENG Z, et al. NFLAT: nonflat-lattice transformer for chinese named entity recognition[J]. arXiv:2205. 05832, 2022.
[57] LI J, FEI H, LIU J, et al. Unified named entity recognition as word-word relation classification[C]//Proceedings of the 36th AAAI Conference on Artificial Intelligence, 2022: 10965-10973.
[58] MEMORY L S T. Long short-term memory[J]. Neural Computation, 2010, 9(8): 1735-1780.
[59] KIM Y, JERNITE Y, SONTAG D, et al. Character-aware neural language models[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence, 2016: 2741-2749.
[60] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems 27, 2014: 2672-2680.
[61] YU L, ZHANG W, WANG J, et al. SeqGAN: sequence generative adversarial nets with policy gradient[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence, 2016: 2852-2858.
[62] MIKOLOV T, SUTSKEVER I, DEORAS A, et al. Subword language modeling with neural networks[EB/OL]. (2011) [2023-11-04]. https://api.semanticscholar.org/CorpusID: 46542477.
[63] LAMPLE G, OTT M, CONNEAU A, et al. Phrase-based & neural unsupervised machine translation[J]. arXiv:1804. 07755, 2018.
[64] LIU Y, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized BERT pretraining approach[J]. arXiv:1907.11692, 2019.
[65] LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition[J]. arXiv: 1603.01360, 2016.
[66] JANSSON P, LIU S. Distributed representation, LDA topic modelling and deep learning for emerging named entity recognition from social media[C]//Proceedings of the 3rd Workshop on Noisy User-generated Text, 2017: 154-159.
[67] AGUILAR G, MAHARJAN S, LóPEZ-MONROY A P, et al. A multi-task approach for named entity recognition in social media data[J]. arXiv:1906.04135, 2019.
[68] GONG C, TANG J, LI Z. An end-to-end named entity recognition model for Chinese[J]. IOP Publishing, 2019, 692(1): 012050.
[69] LI M, ZHANG Y, HUANG M, et al. Named entity recognition in Chinese electronic medical record using attention mechanism[C]//Proceedings of the 2019 International Conference on Internet of Things and IEEE Green Computing and Communications and IEEE Cyber, Physical and Social Computing and IEEE Smart Data, 2019: 649-654.
[70] GUI T，MA R，ZHANG Q，et al. Rethinking CNN-based Chinese NER and dictionary[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, 2019: 4982-4988.
[71] YE N, QIN X, DONG L, et al. Chinese named entity recognition based on character-word vector fusion[J]. Wireless Communications and Mobile Computing, 2020. DOI:10.1155/ 2020/8866540.
[72] LAN Y, XU H, XU K. Research on named entity recognition for science and technology terms in Chinese based on dependent entity word vector[C]//Proceedings of the 2020 IEEE 14th International Conference on Anti-counterfeiting, Security, and Identification, 2020: 26-31.
[73] GAO Y, WANG Y, WANG P, et al. Medical named entity extraction from Chinese resident admit notes using character and word attention-enhanced neural network[J]. International Journal of Environmental Research and Public Health, 2020, 17(5): 1614.
[74] ZHOU C, ZHAO J, REN C. SUDIR: an approach of sensing urban text data from internet resources based on deep learning[J]. IEEE Access, 2020, 8: 214454-214468.
[75] HAN X, ZHOU F, HAO Z, et al. MAF-CNER: a Chinese named entity recognition model based on multifeature adaptive fusion[J]. Complexity, 2021. DOI:10.1155/2021/6696064.
[76] ZHAI S, GOU D, WANG H, et al. Chinese named entity recognition based on BERT and neural network[C]//Proceedings of the 2021 International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery. Cham: Springer, 2021: 1331-1339.
[77] 赵丹丹, 黄德根, 孟佳娜，等. 多头注意力与字词融合的中文命名实体识别[J]. 计算机工程与应用, 2022, 58(7): 142-149.
ZHAO D D, HUANG D G, MENG J N, et al. Chinese named entity recognition by integrating multi-heads attention mechanism and character and words fusion[J]. Computer Engineering and Applications, 2022, 58(7): 142-149.
[78] 袁健, 章海波. 多粒度融合嵌入的中文实体识别模型[J]. 小型微型计算机系统, 2022, 43(4): 741-746.
YUAN J, ZHANG H B. Chinese entity recognition model of multi-granularity fusion embedded[J]. Journal of Chinese Computer Systems, 2022, 43(4): 741-746.
[79] GAN L, HUANG C. A Chinese named entity recognition method combined with relative position information[C]//Proceedings of the 2021 Asia-Pacific Conference on Communications Technology and Computer Science, 2021: 250-254.
[80] 田雨, 张桂平, 蔡东风, 等. 基于多颗粒度文本表征的中文命名实体识别方法[J]. 中文信息学报, 2022, 36(4): 90-99.
TIAN Y, ZHANG G P, CAI D F, et al. Chinese named entity recognition based on text representation multi-granularty[J]. Journal of Chinese Information Processing, 2022, 36(4): 90-99.
[81] 陈雪松, 朱鑫海, 王浩畅. 基于PMV-LSTM的中文医学命名实体识别[J]. 计算机工程与设计, 2022, 43(11): 3257-3263.
CHEN X S, ZHU X H, WANG H C. Chinese medical named entity recognition based on PMV-LSTM[J]. Computer Engineering and Design, 2022, 43(11): 3257-3263.
[82] BAO Z, WANG S. Chinese named entity recognition method based on BERT and fusion attention mechanism[C]//Proceedings of the 2022 International Conference on Electronic Information Engineering and Computer Communication, 2022: 80-85.
[83] 封红旗, 孙杨, 杨森, 等. 基于BERT的中文电子病历命名实体识别[J]. 计算机工程与设计, 2023, 44(4): 1220-1227.
FENG H Q, SUN Y, YANG S, et al. Named entity recognition for Chinese electronic medical records based on BERT[J]. Computer Engineering and Design, 2023, 44(4): 1220-1227.
[84] SONG X, YU H, LI S, et al. Robust Chinese named entity recognition based on fusion graph embedding[J]. Electronics, 2023, 12(3): 569.
[85] XU J, WANG L, XU J, et al. Entity extraction based on the parts of speech attention mechanism[C]//Proceedings of the 5th International Conference on Computer Information Science and Artificial Intelligence, 2023: 591-596.
[86] ZHANG D, ZHENG G, LIU H, et al. AWdpCNER: automated WDP Chinese named entity recognition from wheat diseases and pests text[J]. Agriculture, 2023, 13(6): 1220.
[87] SOUZA F, NOGUEIRA R, LOTUFO R. Portuguese named entity recognition using BERT-CRF[J]. arXiv:1909.10649, 2019.
[88] CAI R, QIN B, CHEN Y, et al. Sentiment analysis about investors and consumers in energy market based on BERT-BiLSTM[J]. IEEE Access, 2020, 8: 171408-171415.
[89] GAO W, ZHENG X, ZHAO S. Named entity recognition method of Chinese EMR based on BERT-BiLSTM-CRF[J]. Journal of Physics: Conference Series, 2021, 1848: 012083.
[90] LI D, TU Y, ZHOU X, et al. End-to-end Chinese entity recognition based on BERT-BiLSTM-ATT-CRF[J]. ZTE Communications, 2022, 20(S1): 27.
[91] TANG X, HUANG Y, A M, et al. A multi-task BERT-BiLSTM-AM-CRF strategy for Chinese named entity recognition[J]. Neural Processing Letters, 2023, 55(2): 1209-1229.
[92] BROWN T, MANN B, RYDER N, et al. Language models are few-shot learners[C]//Advances in Neural Information Processing Systems 33, 2020: 1877-1901.
[93] CUI Y, YANG Z, LIU T. PERT: pre-training BERT with permuted language model[J]. arXiv:2203.06906, 2022.
[94] LIU J, CHU X, WANG J, et al. Combining permuted language model and adversarial training for Chinese machine reading comprehension[J]. Journal of Intelligent & Fuzzy Systems, 2024, 46(4): 10059-10073.
[95] 刘哲, 张文学. 基于乱序语言模型字嵌入的医疗命名实体识别方法分析[J]. 电子技术, 2022, 51(11): 32-36.
LIU Z, ZHANG W X. Medical named entity recognition method based on word embedding of out-of-order language model[J]. Electronic Technology, 2022, 51(11): 32-36.
[96] WADDEN D, WENNBERG U, LUAN Y, et al. Entity, relation, and event extraction with contextualized span representations[J]. arXiv:1909.03546, 2019.
[97] EBERTS M, ULGES A. Span-based joint entity and relation extraction with transformer pre-training[J]. arXiv:1909. 07755, 2019.
[98] LEE K, HE L, LEWIS M, et al. End-to-end neural coreference resolution[J]. arXiv:1707.07045, 2017.
[99] KATIYAR A, CARDIE C. Nested named entity recognition revisited[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018: 861-871.
[100] SOHRAB M G, MIWA M. Deep exhaustive model for nested named entity recognition[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018: 2843-2849.
[101] HUANG Z, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging[J]. arXiv:1508.01991, 2015.
[102] 赵继贵, 钱育蓉, 王魁, 等. 中文命名实体识别研究综述[J]. 计算机工程与应用, 2024, 60(1): 15-27.
ZHAO J G, QIAN Y R, WANG K, et al. Survey of Chinese named entity recognition research[J]. Computer Engineering and Application, 2024, 60(1): 15-27.
[103] ZHENG S, HAO Y, LU D, et al. Joint entity and relation extraction based on a hybrid neural network[J]. Neurocomputing, 2017, 257: 59-66.
[104] SUI D, ZENG X, CHEN Y, et al. Joint entity and relation extraction with set prediction networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023.
[105] WANG C, WANG H, ZHUANG H, et al. Chinese medical named entity recognition based on multi-granularity semantic dictionary and multimodal tree[J]. Journal of Biomedical Informatics, 2020, 111: 103583.
[106] XU B, HUANG S, SHA C, et al. MAF: a general matching and alignment framework for multimodal named entity recognition[C]//Proceedings of the 15th ACM International Conference on Web Search and Data Mining, 2022: 1215-1223.
[107] CHEN M, LUO X, SHEN H, et al. A Chinese nested named entity recognition approach using sequence labeling[J]. International Journal of Web Information Systems, 2023, 19(1): 42-60.
[108] WANG Y, TONG H, ZHU Z, et al. Nested named entity recognition: a survey[J]. ACM Transactions on Knowledge Discovery from Data, 2022, 16(6): 1-29.
[109] KANG H, XIAO J, ZHANG Y, et al. A research toward Chinese named entity recognition based on transfer learning[J]. International Journal of Computational Intelligence Systems, 2023, 16(1): 56.
[110] GUAN G, ZHU M. New research on transfer learning model of named entity recognition[J]. Journal of Physics: Conference Series, 2019, 1267(1): 012017.
[111] GUI T, ZOU Y, ZHANG Q, et al. A lexicon-based graph neural network for Chinese NER[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 1040-1050.