结合对抗训练和注意力机制的蔬菜种植领域命名实体识别

doi:10.3778/j.issn.1002-8331.2311-0048

摘要/Abstract

摘要： 针对复杂语境下的蔬菜种植领域命名实体识别任务中存在实体分布不均衡、实体边界不清晰和语义关联不足等问题，提出一种基于对抗训练和多头自注意力机制的蔬菜种植领域命名实体识别模型。以番茄为研究对象，采用ALBERT（a lite BERT）提取语料动态词向量，结合对抗训练对词向量扰动生成对抗样本并集成为嵌入层输出，缓解农业数据不平衡问题；在特征提取层中通过引入多头自注意力机制对BiLSTM提取的序列特征进一步优化权重分布，更多关注边界信息，加强文本语义关联；最后采用条件随机场解码标注序列。在由8个类别和5 542条标注样本构建的语料库Veg-Tomato上进行了实验。结果表明，该模型的精确率、召回率和F1值分别达89.26%、85.77%、87.48%，较最优基线模型提高了0.69、3.56、2.21个百分点，在小样本数据上仍能表现较高的识别精度，适用于蔬菜种植领域命名实体识别任务。

关键词: 蔬菜种植, 命名实体识别, ALBERT, 对抗训练, 多头自注意力

Abstract: A vegetable planting domain named entity recognition model based on adversarial training and multi-head self-attention mechanism is proposed to solve the issues of entity imbalance, unclear entity boundaries and insufficient semantic association in the task of named entity recognition in the vegetable planting field under complex contexts. It utilizes ALBERT (A Lite BERT) to extract dynamic word vectors for tomato as the research object. The word vectors are perturbed using adversarial training to generate adversarial samples, which are then integrated into the embedding layer output. The sequence features extracted by the BiLSTM are further optimized using the multi-head self-attention mechanism in the feature extraction layer. This optimization enhances the weight distribution, focuses more on boundary information and strengthens the semantic association of the text, thereby alleviating the imbalance of agricultural data and improving the robustness of the model. The labeled sequences are decoded using a conditional random field. Experimental evaluations are conducted on the Veg-Tomato corpus, consisting of 8 categories and 5 542 annotated samples. The results demonstrate that VC-AMS achieves precision, recall, and F1 values of 89.26%, 85.77%, and 87.48% respectively, surpassing the optimal baseline model by 0.69, 3.56, and 2.21 percentage points. The model exhibits high recognition accuracy even on small sample data, making it suitable for named entity recognition tasks in the vegetable planting domain.

Key words: vegetable cultivation, named entity recognition, a lite BERT(ALBERT), adversarial training, multi-head self-attention

胡乔, 赵春江, 吴华瑞, 缪祎晟, 郭旺. 结合对抗训练和注意力机制的蔬菜种植领域命名实体识别[J]. 计算机工程与应用, 2025, 61(9): 343-352.

HU Qiao, ZHAO Chunjiang, WU Huarui, MIAO Yisheng, GUO Wang. Named Entity Recognition in Vegetable Cultivation Combining Adversarial Training and Attention Mechanism[J]. Computer Engineering and Applications, 2025, 61(9): 343-352.

参考文献

[1] WANG M, ZHOU T, WANG H H, et al. Chinese power dispatching text entity recognition based on a double-layer BiLSTM and multi-feature fusion[J]. Energy Reports, 2022, 8: 980-987.
[2] SRIVASTAVA S, PAUL B, GUPTA D. Study of word embeddings for enhanced cyber security named entity recognition[J]. Procedia Computer Science, 2023, 218: 449-460.
[3] ZHANG S X, ZHU H Y, XU H Q, et al. A named entity recognition method towards product reviews based on BiLSTM-attention-CRF[J]. International Journal of Computational Science and Engineering, 2022, 25(5): 479.
[4] 赵丹丹, 黄德根, 孟佳娜, 等. 多头注意力与字词融合的中文命名实体识别[J]. 计算机工程与应用, 2022, 58(7): 142-149.
ZHAO D D, HUANG D G, MENG J N, et al. Chinese named entity recognition by integrating multi-heads attention mechanism and character and words fusion[J]. Computer Engineering and Applications, 2022, 58(7): 142-149.
[5] LI D Y, YAN L, YANG J Z, et al. Dependency syntax guided BERT-BiLSTM-GAM-CRF for Chinese NER[J]. Expert Systems with Applications, 2022, 196: 116682.
[6] PARK Y J, LEE M A, YANG G J, et al. Biomedical text NER tagging tool with web interface for generating BERT-based fine-tuning dataset[J]. Applied Sciences, 2022, 12(23): 12012.
[7] 姚蕾, 蒋明峰, 方贤, 等. 结合部首特征和BERT-Transformer-CRF的中文电子病历实体识别方法研究[J]. 软件工程, 2022, 25(12): 30-36.
YAO L, JIANG M F, FANG X, et al. Research on Chinese clinical named entity recognition method based on radical feature and BERT-transformer-CRF[J]. Software Engineering, 2022, 25(12): 30-36.
[8] GUAN G L, ZHU M. New research on transfer learning model of named entity recognition[J]. Journal of Physics: Conference Series, 2019, 1267(1): 012017.
[9] 赵鹏飞, 赵春江, 吴华瑞, 等. 基于注意力机制的农业文本命名实体识别[J]. 农业机械学报, 2021, 52(1): 185-192.ZHAO P F, ZHAO C J, WU H R, et al. Named entity recognition of Chinese agricultural text based on attention mechanism[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(1): 185-192.
[10] 韦紫君, 宋玲, 胡小春, 等. 基于实体级遮蔽BERT与BiLSTM-CRF的农业命名实体识别[J]. 农业工程学报, 2022, 38(15): 195-203.
WEI Z J, SONG L, HU X C, et al. Named entity recognition of agricultural based entity-level masking BERT and BiLSTM-CRF[J]. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(15): 195-203.
[11] 刘合兵, 张德梦, 熊蜀峰, 等. 融合ALBERT与规则的小麦病虫害命名实体识别[J]. 计算机科学与探索, 2023, 17(6): 1395-1404.
LIU H B, ZHANG D M, XIONG S F, et al. Named entity recognition of wheat diseases and pests fusing ALBERT and rules[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(6): 1395-1404.
[12] 蒲攀, 张越, 刘勇, 等. Transformer优化及其在苹果病虫命名实体识别中的应用[J]. 农业机械学报, 2023, 54(6): 264-271.
PU P, ZHANG Y, LIU Y, et al. Transformer optimization and application in named entity recognition of apple diseases and pests[J]. Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(6): 264-271.
[13] 李书琴, 张明美, 刘斌. 融合字词语义信息的猕猴桃种植领域命名实体识别研究[J]. 农业机械学报, 2022, 53(12): 323-331.
LI S Q, ZHANG M M, LIU B. Kiwifruit planting entity recognition based on character and word information fusion[J]. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53(12): 323-331.
[14] 陈晓晋, 唐球, 王耀君. 面向农业知识图谱构建的文本实体标注准则构建及应用[J]. 电子技术应用, 2023, 49(5): 1-7.
CHEN X J, TANG Q, WANG Y J. Construction and application of agricultural text data entity labeling criteria for agricultural knowledge graph construction[J]. Application of Electronic Technique, 2023, 49(5): 1-7.
[15] 赵鹏飞, 赵春江, 吴华瑞, 等. 基于BERT的多特征融合农业命名实体识别[J]. 农业工程学报, 2022, 38(3): 112-118.ZHAO P F, ZHAO C J, WU H R, et al. Recognition of the agricultural named entities with multi-feature fusion based on BERT[J]. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(3): 112-118.
[16] LAN Z Z , CHEN M D, GOODMAN S ,et al. ALBERT: a lite BERT for self-supervised learning of language representations[C]//Proceedings of the International Conference on Learning Representations, 2020.
[17] MIYATO T, DAI A M, GOODFELLOW I. Adversarial training methods for semi-supervised text classification[J]. arXiv:1605.07725, 2016.
[18] LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition[J]. arXiv:1603.01360, 2016.
[19] VASWANI A, SHAZEER N M, PARMAR N, et al. Attention is all you need[J]. arXiv:1706.03762, 2023.
[20] LI J, SUN A X, HAN J L, et al. A survey on deep learning for named entity recognition[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 34(1): 50-70.
[21] HUANG Z H, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging[J].arXiv:1508.01991, 2015.
[22] ZHANG S X, ZHAO M. Chinese agricultural diseases named entity recognition based on BERT-CRF[C]//Proceedings of the 2020 5th International Conference on Mechanical, Control and Computer Engineering. Piscataway: IEEE, 2020: 1148-1151.
[23] HE B J, CHEN J F. Named entity recognition method in network security domain based on BERT-BiLSTM-CRF[C]//Proceedings of the 2021 IEEE 21st International Conference on Communication Technology. Piscataway: IEEE, 2021: 508-512.
[24] LOU Q F, WANG S T, CHEN J H, et al. Named entity recognition of traditional Chinese medicine cases based on RoBERTa-BiLSTM-CRF[C]//Proceedings of the 2023 IEEE International Conference on Bioinformatics and Biomedicine. Piscataway: IEEE, 2023: 4609-4614.