Semantic and Syntactic Features with Multi-Attentive Interaction for Medical Question Answering

doi:10.3778/j.issn.1002-8331.2102-0172

Abstract

Abstract: For Chinese medical question answering task, in order to capture important syntactic and semantic information in question and answer sentences, it is proposed to introduce graph convolutional network to capture syntactic information, and add a multi-attentive pooling module to realize joint learning of word order and syntactic features of question and answer sentences. On the basis of the BERT model learning the high-order semantic features of question and answer sentences, the bi-directional gated recurrent unit is used to describe the global semantic features of the sentence, and the introduction of the graph convolutional neural network to encode the grammatical structure information of the sentence to present a complementary relationship with the sequence features obtained by the bi-directional gated recurrent unit. Through the multi-attentive pooling module, the encoding vectors in different semantic spaces of the question and answer pairs are interacted in pairs, and the co-occurrence characteristics of the question and answer pairs are emphasized. By measuring matching score of the question and answer pairs to find the best answer. The experimental results show that on the cMedQA v1.0 and cMedQA v2.0 data sets, the ACC@1 of the proposed method is improved compared with mainstream deep learning methods. Experiments prove that the introduction of the integrated algorithm of graph convolutional network and multi-attentive pooling module can effectively improve the performance of question answering model.

Key words: question answering, bi-directional gated recurrent unit, graph convolutional network, syntactic information, multi-attentive pooling

摘要： 针对中文医疗自动问答任务，为了捕捉问答句中重要的句法信息和语义信息，提出引入图卷积神经网络捕捉句法信息，并添加多注意力池化模块实现问答句的语序特征和句法特征联合学习的方法。在BERT模型学习问答句的高阶语义特征基础上，利用双向门控循环单元描述句子的全局语义特征，以及引入图卷积神经网络编码句子的语法结构信息，以与双向门控循环单元所获取的序列特征呈现互补关系；通过多注意力池化模块对问答对的不同语义空间上的编码向量进行两两交互，并着重突出问答对的共现特征；通过衡量问答对的匹配分数，找出最佳答案。实验结果表明，在cMedQA v1.0和cMedQA v2.0数据集上，相比于主流的深度学习方法，所提方法的ACC@1有所提高。实验证明引入图卷积神经网络和多注意力池化模块的集成算法能有效提升自动问答模型的性能。

关键词: 自动问答, 双向门循环单元, 图卷积神经网络, 句法信息, 多注意力池化

ZHANG Huali, KANG Xiaodong, LI Xiaojun, LIU Hanqing, WANG Xiaotian. Semantic and Syntactic Features with Multi-Attentive Interaction for Medical Question Answering[J]. Computer Engineering and Applications, 2022, 58(18): 233-240.

张华丽, 康晓东, 李小军, 刘汉卿, 王笑天. 语义及句法特征多注意力交互的医疗自动问答[J]. 计算机工程与应用, 2022, 58(18): 233-240.

References

[1] 王智悦，于清，王楠，等.基于知识图谱的智能问答研究综述[J].计算机工程与应用，2020，56（23）：1-11.
WANG Z Y，YU Q，WANG N，et al.Summary of intelligent question answering research based on knowledge graph[J].Computer Engineering and Applications，2020，56（23）：1-11.
[2] JAIN S，DODIYA T.Rule based architecture for medical question answering system[C]//Proceedings of the Second International Conference on Soft Computing for Problem Solving（SocProS 2012），December 28-30，2012.
[3] CAIRNS B L，NIELSEN R D，MASANZ J J，et al.The MiPACQ clinical question answering system[C]//AMIA Annual Symposium Proceedings，2011：171.
[4] JAIN S，DODIYA T.Rule based architecture for medical question answering system[C]//Proceedings of the Second International Conference on Soft Computing for Problem Solving（SocProS 2012），December 28-30，2012.New Delhi：Springer，2014：1225-1233.
[5] FENG M，BING X，GLASS M R，et al.Applying deep learning to answer selection：a study and an open task[C]//IEEE Automatics Speech Recognition and Understanding Workshop，2015.
[6] TAN M，SANTOS C D，XIANG B，et al.LSTM-based deep learning models for non-factoid answer selection[J].arXiv：1511.04108，2015.
[7] ZHANG S，ZHANG X，WANG H，et al.Chinese medical question answer matching using end-to-end character-level multi-scale CNNs[J].Applied Sciences，2017，7（8）：767.
[8] ZHANG Y，LU W，OU W，et al.Chinese medical question answer selection via hybrid models based on CNN and GRU[J].Multimedia Tools and Applications，2020，79：14751-14776.
[9] 邵曦，陈明.结合Bi-LSTM和注意力模型的问答系统研究[J].计算机应用与软件，2020，37（10）：52-56.
SHAO X，CHEN M.A question and answering based on Bi-LSTM and attention model[J].Computer Applications and Software，2020，37（10）：52-56.
[10] ZHANG S，ZHANG X，WANG H，et al.Multi-scale attentive interaction networks for Chinese medical question answer selection[J].IEEE Access，2018.
[11] CUI X，HAN J.Chinese medical question answer matching based on interactive sentence representation learning[J].arXiv：2011.13573，2020.
[12] TAI K S，SOCHER R，MANNING C D.Improved semantic representations from tree-structured long short-term memory networks[J].Computer Science，2015，5（1）：36.
[13] ZHANG Y，GUO Z，LU W.Attention guided graph convolutional networks for relation extraction[J].arXiv：1906.
07510，2019.
[14] 张军莲，张一帆，汪鸣泉，等.基于图卷积神经网络的中文实体关系联合抽取[J].计算机工程，2021，47（12）：103-111.
ZHANG J L，ZHANG Y F，WANG M Q，et al.Joint extraction of Chinese entity relations based on graph convolutional neural network[J].Computer Engineering，2021，47（12）：103-111.
[15] 王晓霞，钱雪忠，宋威.基于注意力与图卷积网络的关系抽取模型[J].计算机应用，2021，41（2）：350-356.
WANG X X，QIAN X Z，SONG W.Relation extraction model via attention-based graph convolutional network[J].Journal of Computer Applications，2021，41（2）：350-356.
[16] DEVLIN J，CHANG M W，LEE K，et al.Bert：pre-training of deep bidirectional transformers for language understanding[J].arXiv：1810.04805，2018.
[17] 石春丹，秦岭.基于BGRU-CRF的中文命名实体识别方法[J].计算机科学，2019，46（9）：237-242.
SHI C D，QIN L.Chinese named entity recognition method based on BGRU-CRF[J].Computer Science，2019，46（9）：237-242.
[18] CHEN Y，GEL Y R，AVRACHENKOV K.LFGCN：levitating over graphs with levy flights[J].arXiv：2009.02365，2020.
[19] SANTOS C D，TAN M，XIANG B，et al.Attentive pooling networks[J].arXiv：1602.03609，2016.
[20] 张胜.基于深度神经网络的中文医疗社区自动问答技术研究[D].长沙：国防科技大学，2017.
ZHANG S.Research on Chinese medical question answering based on deep neural network[D].Changsha：National University of Defense Technology，2017.
[21] 王杰.基于深度神经网络的中文医疗社区问答匹配技术研究[D].成都：电子科技大学，2020.
WANG J.Research on question and answer matching technology of Chinese medical community based on deep neural network[D].Chengdu：University of Electronic Science and Technology of China，2020.