融合空间位置注意力机制的英语题注生成模型

doi:10.3778/j.issn.1002-8331.2111-0300

计算机工程与应用 ›› 2022, Vol. 58 ›› Issue (12): 139-148.DOI: 10.3778/j.issn.1002-8331.2111-0300

融合空间位置注意力机制的英语题注生成模型

王琴，王鑫，颜靖柯，钟美玲，曾静

1.桂林电子科技大学基础教学部，广西北海 536000
2.桂林电子科技大学计算机与信息安全学院，广西桂林 541004
3.桂林电子科技大学海洋工程学院，广西北海 536000
4.桂林电子科技大学计算机工程学院，广西北海 536000
5.电子科技大学信息与软件工程学院，成都 610000

出版日期:2022-06-15 发布日期:2022-06-15

English Caption Generation Model Fused with Attention Mechanism of Spatial Position

WANG Qin, WANG Xin, YAN Jingke, ZHONG Meiling, ZENG Jing

1.Basic Teaching Department, Guilin University of Electronic Technology, Beihai, Guangxi 536000, China
2.School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi 541004, China
3.School of Marine Engineering, Guilin University of Electronic Technology, Beihai, Guangxi 536000, China
4.College of Computer Engineering, Guilin University of Electronic Technology, Beihai, Guangxi 536000, China
5.School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu 610000, China

Online:2022-06-15 Published:2022-06-15

摘要/Abstract

摘要： 为使题注生成模型生成流畅、连贯和信息丰富的特定信息题注，在Transformer架构的基础上提出了Transformer Chart to Text（TransChartText）模型。通过筛选各种科研论文和新闻文章网站，制作了基于图表的题注描述数据集，该数据集的英语题注描述涵盖了丰富的数据类别和逻辑推理。引入数据变量替换图表数据值，有效提高了模型生成题注的内容选择，促使模型生成了连贯的题注内容。为进一步增强模型学习词与词之间位置关系的能力并降低错误词序频率，模型分别对编码器和解码器引入空间位置嵌入编码和集束搜索算法。实验结果表明，TransChartText模型在内容选择（CS）、内容排序（CO）、ROUGE、BLEU指标上取得了更好的分数，生成了高质量的基于图表的英语题注。

关键词: 语言模型, 生成式题注, Transformer, 注意力机制, 集束搜索

Abstract: The transformer chart to text（TransChartText） model is proposed based on Transformer architecture in order to make the English caption generation model generate fluent, coherent and informative specific information annotations. By screening various scientific research papers and news article websites, a chart-based annotation description data set is made, which covers a wealth of data categories and logical reasoning. Data variables are introduced to replace the data values of the graph, which effectively improves the selection of the content of the captions generated by the model and promoted the model to generate coherent captions. In order to enhance the ability of learning the position relation between words and reduce the frequency of wrong word order, the spatial position embedding coding and cluster search algorithm are introduced into the encoder and decoder respectively. Experimental results show that TransChartText model achieves better scores on content selection（CS）, content sequencing（CO）, ROUGE and BLEU, and generates high-quality chart-based English captions.

Key words: language model, generative caption, Transformer, attention mechanism, beam search

王琴, 王鑫, 颜靖柯, 钟美玲, 曾静. 融合空间位置注意力机制的英语题注生成模型[J]. 计算机工程与应用, 2022, 58(12): 139-148.

WANG Qin, WANG Xin, YAN Jingke, ZHONG Meiling, ZENG Jing. English Caption Generation Model Fused with Attention Mechanism of Spatial Position[J]. Computer Engineering and Applications, 2022, 58(12): 139-148.

参考文献

[1] CARBERRY S，ELZER S，DEMIR S.Information graphics：an untapped resource for digital libraries[C]//Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.Seattle：ACM，2006：581-588.
[2] DUBOUE P A，MCKEOWN K.Statistical acquisition of content selection rules for natural language generation[C]//Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing，2003：121-128.
[3] STENT A，PRASAD R，WALKER M.Trainable sentence planning for complex information presentations in spoken dialog systems[C]//Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics（ACL-04）.Barcelona：ACL，2004：79-86.
[4] WISEMAN S，SHIEBER S M，RUSH A M.Challenges in data-to-document generation[J].arXiv：1707.08052，2017.
[5] LI G，CREGO J M，SENELLART J.Enhanced transformer model for data-to-text generation[C]//Proceedings of the 3rd Workshop on Neural Generation and Translation.Hong Kong：Association for Computational Linguistics，2019：148-156.
[6] 郭望皓，范江威，张克亮.融合语言学知识的神经机器翻译研究进展[J].计算机科学与探索，2021，15（7）：1183-1194.
GUO W H，FAN J W，ZHANG K L.Advance research on neural machine translation integrating linguistic knowledge[J].Journal of Frontiers of Computer Science and Technology，2021，15（7）：1183-1194.
[7] AI X，FANG B.Empirical regularization for synthetic sentence pairs in unsupervised neural machine translation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Virtual Event：AAAI，2021：12471-12479.
[8] VIJAYAKUMAR A，COGSWELL M，SELVARAJU R，et al.Diverse beam search for improved description of complex scenes[C]//Proceedings of the AAAI Conference on Artificial Intelligence.New Orleans，Louisian：AAAI，2018：7371-7379.
[9] CUI Z，BADAM S K，YAL?IN M A，et al.Datasite：proactive visual data exploration with computation of insight-based recommendations[J].Information Visualization，2019，18（2）：251-267.
[10] WANG Y，SUN Z，ZHANG H，et al.DataShot：automatic generation of fact sheets from tabular data[J].IEEE Transactions on Visualization and Computer Graphics，2019，26（1）：895-905.
[11] SRINIVASAN A，DRUCKER S M，ENDERT A，et al.Augmenting visualizations with interactive data facts to facilitate interpretation and communication[J].IEEE Transactions on Visualization and Computer Graphics，2018，25（1）：672-681.
[12] CHEN C，ZHANG R，KOH E，et al.Figure captioning with reasoning and sequence-level training[J].arXiv：1906.02850，2019.
[13] PUDUPPULLY R，DONG L，LAPATA M.Data-to-text generation with content selection and planning[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Honolulu，Hawaii：AAAI，2019：6908-6915.
[14] AOKI K，MIYAZAWA A，ISHIGAKI T，et al.Controlling contents in data-to-document generation with human-designed topic labels[J].Computer Speech & Language，2021，66：101154.
[15] 周龙，王晨，史崯.基于RNN的Webshell检测研究[J].计算机工程与应用，2020，56（14）：88-92.
ZHOU L，WANG C，SHI Y.Research on Webshell detection based on RNN[J].Computer Engineering and Applications，2020，56（14）：88-92.
[16] 丁宇阳，李明悦，谢柠宇，等.双LSTM的光场图像去雨算法研究[J].计算机工程与应用，2021，57（18）：227-237.
DING Y Y，LI M R，XIE N Y，et al.Research of dual LSTM methocl for rain streaks removal on light field images[J].Computer Engineering and Applications，2021，57（18）：227-237.
[17] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[J].arXiv：1706.03762，2017.
[18] CHEN W，CHEN J，SU Y，et al.Logical natural language generation from open-domain tables[J].arXiv：2004.10404，
2020.
[19] PARIKH A P，WANG X，GEHRMANN S，et al.Totto：a controlled table-to-text generation dataset[J].arXiv：2004.14373，
2020.
[20] NOZZA D，MANCHANDA P，FERSINI E，et al.Learning to adapt with word embeddings：domain adaptation of named entity recognition systems[J].Information Processing & Management，2021，58（3）：102537.
[21] TAN Z，WANG M，XIE J，et al.Deep semantic role labeling with self-attention[C]//Proceedings of the AAAI Conference on Artificial Intelligence.New Orleans，Louisian：AAAI，2018：4929-4936.
[22] WANG S，CLARK R，WEN H，et al.End-to-end，sequence-to-sequence probabilistic visual odometry through deep neural networks[J].The International Journal of Robotics Research，2018，37（4/5）：513-542.
[23] LEBRET R，GRANGIER D，AULI M.Neural text generation from structured data with application to the biography domain[J].arXiv：1603.07771，2016.
[24] PUDUPPULLY R，DONG L，LAPATA M.Data-to-text generation with entity modeling[J].arXiv：1906.03221，2019.

融合空间位置注意力机制的英语题注生成模型

English Caption Generation Model Fused with Attention Mechanism of Spatial Position

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	张昕, 刘思远, 徐雁翎. 结合注意力机制的知识感知推荐算法[J]. 计算机工程与应用, 2022, 58(9): 168-174.
[2]	胡章芳, 蹇芳, 唐珊珊, 明子平, 姜博文. DFSMN-T：结合强语言模型Transformer的中文语音识别[J]. 计算机工程与应用, 2022, 58(9): 187-194.
[3]	杨永波, 李栋. 改进YOLOv5的轻量级安全帽佩戴检测算法[J]. 计算机工程与应用, 2022, 58(9): 201-207.
[4]	王玲敏, 段军, 辛立伟. 引入注意力机制的YOLOv5安全帽佩戴检测方法[J]. 计算机工程与应用, 2022, 58(9): 303-312.
[5]	高广尚. 深度学习推荐模型中的注意力机制研究综述[J]. 计算机工程与应用, 2022, 58(9): 9-18.
[6]	何倩倩, 孙静宇, 曾亚竹. 基于邻域感知图神经网络的会话推荐[J]. 计算机工程与应用, 2022, 58(9): 107-115.
[7]	孙刘杰, 赵进, 王文举, 张煜森. 多尺度Transformer激光雷达点云3D物体检测[J]. 计算机工程与应用, 2022, 58(8): 136-146.
[8]	赵丹丹, 黄德根, 孟佳娜, 谷丰, 张攀. 多头注意力与字词融合的中文命名实体识别[J]. 计算机工程与应用, 2022, 58(7): 142-149.
[9]	陈秋嫦, 赵晖, 左恩光, 赵玉霞, 魏文钰. 上下文感知的树递归神经网络下隐式情感分析[J]. 计算机工程与应用, 2022, 58(7): 167-175.
[10]	王欣, 王美丽, 边党伟. 融合MobileNetv2和注意力机制的轻量级人像分割算法[J]. 计算机工程与应用, 2022, 58(7): 220-228.
[11]	刘文婷, 卢新明. 基于计算机视觉的Transformer研究进展[J]. 计算机工程与应用, 2022, 58(6): 1-16.
[12]	张明, 卢庆华, 黄元忠, 李瑞轩. 自然语言语法纠错的最新进展和挑战[J]. 计算机工程与应用, 2022, 58(6): 29-41.
[13]	陈佐瓒, 徐兵, 丁小军, 甘井中. 基于Encoder-Decoder框架的双监督机制自然场景文本识别[J]. 计算机工程与应用, 2022, 58(6): 128-133.
[14]	曹渝昆, 孙涛. 基于GLSTM和Attention的中文事件要素提取[J]. 计算机工程与应用, 2022, 58(6): 157-163.
[15]	李彦辰, 张小俊, 张明路, 沈亮屹. 基于改进Efficientdet的自动驾驶场景目标检测[J]. 计算机工程与应用, 2022, 58(6): 183-191.