面向采购文件的跨模态图片文本命名实体识别

doi:10.3778/j.issn.1002-8331.2303-0045

摘要/Abstract

摘要： 智慧供应链的数智化采购环节能够提高采购工作效率，节省大量人力成本。采购文件中包括大量证照资质等文件，针对其中图片文本中文字排版参差不齐、扫描图像不清晰等问题，设计了基于深度学习的端到端跨模态命名实体识别模型O2V2BLC（OCR-Vector-Bi-LSTM-CRF），从图片文本中识别命名实体。该模型针对光学字符识别技术识别出的图片文本字符，定义连续文本字符边界，将边界内每个文本字符映射为向量，设计双向长短期记忆（Bi-LSTM）网络捕获边界内字符序列的上下文语义，计算字符状态分数矩阵，并通过条件随机场约束字符标记序列规则，获得全局最优标记序列。针对训练集计算命名实体预测误差，动态优化O2V2BLC模型的参数，实现命名实体识别。将该方法应用于采购文件资质类型等图片文本数据，能够有效识别图片中的投标单位、专家姓名、专业名称等命名实体，与条件随机场、隐马尔可夫算法、Bert-Bi-LSTM-CRF模型进行对比，显著提高了实体识别准确率，为智慧供应链的数智化采购提供支持。

关键词: 智慧供应链, 命名实体识别, 光学字符识别, 双向长短期记忆网络, 条件随机场

Abstract: The digital and intelligent procurement of smart supply chain can improve the efficiency of procurement and save a lot of labor costs. The procurement documents include a large number of documents such as certificates and qualifications. In view of uneven text layout and unclear scanned images, this paper designs an end-to-end cross-modal named entity recognition model O2V2BLC (OCR-Vector-Bi-LSTM-CRF) based on deep learning to detect named entities from the image. This model defines the continuous text character boundary for the characters recognized by optical character recognition technology, maps each character within the boundary into a vector, designs a bi-directional short and long term memory network (Bi-LSTM) to capture the context semantics of the character sequence within the boundary, calculates the state matrix of character, and obtains the global optimal marker sequence by constraining the character marker sequence rules by conditional random fields. The prediction error of named entities is calculated according to the training set, and the parameters of O2V2BLC model are dynamically optimized. Applying the method in this paper to images such as the qualification type of the procurement document can effectively identify the bidding unit, expert name, professional name and other named entities in the images. Compared with the conditional random field, hidden Markov algorithm and Bert-Bi-LSTM-CRF model, it improves the accuracy of entity identification and provides support for the digital and intelligent procurement of the intelligent supply chain.

Key words: smart supply chain, named entity recognition, optical character recognition, bi-directional , long and short-term memory (Bi-LSTM), conditional random fields

杨赛, 刘昕, 于绍文. 面向采购文件的跨模态图片文本命名实体识别[J]. 计算机工程与应用, 2024, 60(3): 213-219.

YANG Sai, LIU Xin, YU Shaowen. Cross-Model Named Entity Recognition in Pictures for Procurement Documents[J]. Computer Engineering and Applications, 2024, 60(3): 213-219.

参考文献

[1] 李亮. 基于Tesseract_OCR的驾驶证识别系统设计与实现[D]. 成都: 电子科技大学, 2018.
LI L. Design and implementation of driving license identification system based on Tesseract_OCR[D].Chengdu: University of Electronic Science and Technology of China, 2018.
[2] 王康维, 赵磊, 黄鑫炎, 等. 一种低亮度非均匀光照文档图片快速二值化方法[J]. 光电子·激光, 2020, 31(12):1333-1340.
WANG K W, ZHAO L, HUANG X Y, et al.A fast binarization method for dark and uneven illumination docement images[J].Journal of Optoelectronics·Laser, 2020, 31(12):1333-1340.
[3] 吴冰航. 基于机器视觉的工业OCR识别系统的设计与实现[D]. 成都: 电子科技大学, 2021.
WU B H. Design and implementation of industrial OCR recognition system based on machine vision[D]. Chengdu:University of Electronic Science and Technology of China, 2021.
[4] 王凌燕. 中文字符识别系统的研究与实现[J]. 吉林大学学报 (信息科学版), 2020, 38(2):199-205.
WANG L Y. Research and implement of Chinese OCR system[J]. Journal of Jilin University (Information Science Edition), 2020, 38(2):199-205.
[5] ZHAO L, JIA K. Application of CRNN based OCR in health records system[C]//Proceedings of the 3rd International Conference on Multimedia Systems and Signal Processing, 2018: 46-50.
[6] LEE C Y, OSINDERO S. Recursive recurrent nets with attention modeling for OCR in the wild[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 2231-2239.
[7] WICK C, REUL C, PUPPE F. Calamari-a high-performance tensorflow-based deep learning package for optical character recognition[J]. arXiv:1807.02004, 2018.
[8] KOHúT J, HRADI? M. TS-Net: OCR trained to switch between text transcription styles[C]//International Conference on Document Analysis and Recognition. Cham: Springer, 2021: 478-493.
[9] LAY K K, CHO A. Myanmar named entity recognition with hidden Markov model[J]. Published in International Journal of Trend in Scientific Research and Development, 2019, 3(4): 1144-1147.
[10] LEE W J, KIM K, LEE E Y, et al. Conditional random fields for clinical named entity recognition: a comparative study using Korean clinical texts[J]. Computers in Biology and Medicine, 2018, 101:7-14.
[11] ARORA R, TSAI C T, TSERETELI K, et al. A semi-Markov structured support vector machine model for high-precision named entity recognition[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 5862-5866.
[12] ABD M T, MOHD M. A comparative study of word representation methods with conditional random fields and maximum entropy Markov for bio-named entity recognition[J]. Malaysian Journal of Computer Science, 2018, 31(5):15-30.
[13] JIA Y, XU X. Chinese named entity recognition based on CNN-BiLSTM-CRF[C]//2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS), 2018.
[14] SONG Y, TIAN S, YU L. A method for identifying local drug names in Xinjiang based on BERT-BiLSTM-CRF[J]. Automatic Control and Computer Sciences, 2020, 54(3):179-190.
[15] 孙绍丹. 数字人文视域下历史报纸资源语义化知识组织研究[D]. 长春: 吉林大学, 2022.
SUN S D.Research on semantics knowledge organization of historical newspaper resources in digital humanities[D].Changchun: Jinlin University, 2022.
[16] BOROS E, HAMDI A, PONTES E L, et al. Alleviating digitization errors in named entity recognition for historical documents[C]//Proceedings of the 24th Conference on Computational Natural Language Learning, 2020.
[17] 肖连杰, 孟涛, 王伟, 等. 基于深度学习的情报分析方法识别研究*——以安全情报领域为例[J]. 数据分析与知识发现, 2019, 3(10): 20-28.
XIAO L J, MENG T, WANG W, et al.Entity recognition of intelligence method based on deep learning: taking area of security intelligence for example[J]. Data Analysis and Knowledge Discovery, 2019, 3(10): 20-28.
[18] ROUHOU A C, DHIAF M, KESSENTINI Y, et al. Transformer-based approach for joint handwriting and named entity recognition in historical document[J]. Pattern Recognition Letters, 2022, 155: 128-134.
[19] MILLER D, BOISEN S, SCHWARTZ R, et al. Named entity extraction from noisy input: speech and OCR[C]//Sixth Applied Natural Language Processing Conference, 2000: 316-324.
[20] AHONEN E, HYVONEN E. Publishing historical texts on the semantic web-a case study[C]//2009 IEEE International Conference on Semantic Computing, 2009: 167-173.
[21] MD D, CHOWDHURY M, UDAY S I, et al. Named entity recognition in Bengali text using merged hidden Markov model and rule base approach[C]//2019 7th International Conference on Smart Computing & Communications (ICSCC), 2019.
[22] LI Y, SHETTY P, LIU L, et al. BERTifying the hidden Markov model for multi-source weakly supervised named entity recognition[J]. arXiv:2105.12848, 2021.
[23] MAK R M S, BIJAKSANA M A, HUDA A F. Person entity recognition for the Indonesian Qur’an translation with the approach hidden Markov model-viterbi[J]. Procedia Computer Science, 2019, 157: 214-220.
[24] ABD M T, MOHD M. A comparative study of word representation methods with conditional random fields and maximum entropy Markov for bio-named entity recognition[J]. Malaysian Journal of Computer Science, 2018, 31(5):15-30.
[25] SANTOSO J, SETIAWAN E, YUNIARNO E, et al. Hybrid conditional random fields and K-means for named entity recognition on Indonesian news documents[J]. International Journal of Intelligent Engineering and Systems, 2020, 13(3):233-245.
[26] KONKOL M, KONOPíK M. Maximum entropy named entity recognition for Czech language[C]//14th International Conference on Text, Speech and Dialogue, Pilsen, Czech Republic, September 1-5, 2011. Berlin Heidelberg: Springer, 2011: 203-210.
[27] ZHAI Z, NGUYEN D Q, VERSPOOR K. Comparing CNN and LSTM character-level embeddings in BiLSTM-CRF models for chemical and disease named entity recognition[J].arXiv:1808.08450, 2018.
[28] GUI T, MA R, ZHANG Q, et al. CNN-based Chinese NER with lexicon rethinking[C]//Twenty-Eighth International Joint Conference on Artificial Intelligence, 2019.
[29] MA P, JIANG B, LU Z, et al. Cybersecurity named entity recognition using bidirectional long short-term memory with conditional random fields[J]. Tsinghua Science and Technology, 2021, 26(3):259-265.
[30] ZHU Q, LI X, CONESA A, et al. GRAM-CNN: a deep learning approach with local context for named entity recognition in biomedical text[J]. Bioinformatics, 2018, 34(9): 1547-1554.
[31] KONG J, ZHANG L, JIANG M, et al. Incorporating multi-level CNN and attention mechanism for Chinese clinical named entity recognition[J]. Journal of Biomedical Informatics, 2021, 116:103737.
[32] CHE C, ZHOU C, ZHAO H, et al. Fast and effective biomedical named entity recognition using temporal convolutional network with conditional random field[J]. Mathematical Biosciences and Engineering, 2020, 17(4):3553-3566.
[33] DAI Z, WANG X, NI P, et al. Named entity recognition using BERT BiLSTM CRF for Chinese electronic health records[C]//2019 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), 2019.
[34] AGRAWAL A, TRIPATHI S, VARDHAN M, et al. BERT-based transfer-learning approach for nested named-entity recognition using joint labeling[J]. Applied Sciences, 2022, 12(3): 976.
[35] EMMERT-STREIB F. Comparison of text mining models for food and dietary constituent named-entity recognition[J].Machine Learning and Knowledge Extraction, 2022, 41(1): 254-275.
[36] HA H T, HORáK A. Information extraction from scanned invoice images using text analysis and layout features[J]. Signal Processing: Image Communication, 2022, 102: 116601.
[37] ABADIE N, CARLINET E, CHAZALON J, et al. A benchmark of named entity recognition approaches in historical documents application to 19th century French directories[C]//15th IAPR International Workshop on Document Analysis Systems, La Rochelle, France, May 22-25, 2022. Cham: Springer International Publishing, 2022: 445-460.
[38] DUTTA H, GUPTA A. PNRank: unsupervised ranking of person name entities from noisy OCR text[J]. Decision Support Systems, 2022, 152: 113662.
[39] HUYNH V N, HAMDI A, DOUCET A. When to use OCR post?correction for named entity recognition?[C]//International Conference on Asian Digital Libraries. Cham: Springer, 2020: 33-42.