结合BERT与多尺度CNN的民事纠纷问句意图分类

doi:10.3778/j.issn.1002-8331.2105-0369

摘要/Abstract

摘要： 问句意图分类作为问答系统的关键任务之一，其能否正确分类对于后续的问答任务十分重要。针对民事纠纷问句中存在的长短不一、特征分散、种类繁多的问题，以及传统卷积神经网络和词向量的不足，为了准确获取民事纠纷问句意图类别，构建了结合BERT与多尺度CNN的民事纠纷问句意图分类模型。对民事纠纷问句数据集进行预处理；采用BERT预训练模型对问句进行语义编码和语义补充；使用4个不同的卷积通道进行卷积运算，每个卷积通道由不同尺度的卷积核进行卷积，将4种不同尺度的问句特征进行拼接得到多层次问句特征信息；通过全连接层和Softmax对问句进行分类。实验结果表明，所提出的模型在中文民事纠纷问句数据集上取得了87.41%的准确率，召回率、F1值分别达到了87.52%、87.39%，能够有效解决民事纠纷问句意图分类的问题。

关键词: 民事纠纷问句意图分类, BERT, 多尺度CNN, 自然语言问句理解

Abstract: As one of the key tasks of question answering system, the classification of question intention is very important for the following question answering tasks. Aiming at the problems of different lengths, scattered features and various kinds of questions in civil disputes, as well as the shortcomings of traditional convolutional neural network and word vector, in order to accurately obtain the intention category of questions in civil disputes, intent classification of questions in civil disputes combining BERT and multi-scale CNN Model is constructed. Firstly, the data set of questions of civil dispute is preprocessed. Then, the BERT pre-training model is used to encode and supplement the semantic information of the questions. Then, four different convolution channels are used for convolution operation, and each convolution channel is convolved by convolution kernels of different scales. The multi-level question feature information is obtained by combining four different scale question features. Finally, the questions are classified by full connection layer and Softmax layer. The experimental results show that the proposed model achieves 87.41% accuracy on the data set of civil dispute questions, and the recall rate and F1 value reach 87.52% and 87.39%, respectively, which can effectively solve the problem of intention classification of civil dispute questions.

Key words: civil dispute questions intention classification, bidirectional encoder representations from transformers（BERT）, multi-scale CNN, natural language question comprehension

邢义男, 张娜娜. 结合BERT与多尺度CNN的民事纠纷问句意图分类[J]. 计算机工程与应用, 2022, 58(23): 205-213.

XING Yinan, ZHANG Nana. Intent Classification of Questions in Civil Disputes Combining BERT and Multi-Scale CNN Model[J]. Computer Engineering and Applications, 2022, 58(23): 205-213.

参考文献

[1] 朱颂华.常年法律顾问业务的现状与对策[J].法制博览，2020（7）：170-171.
ZHU S H.Current situation and countermeasure of perennial legal counsel business[J].Legality Vision，2020（7）：170-171.
[2] 韩东方，吐尔地·托合提，艾斯卡尔·艾木都拉.问答系统中问句分类方法研究综述[J].计算机工程与应用，2021，57（6）：10-21.
HAN D F，TURDY T，ASKAR H.Survey on question classification method in question answering system[J].Computer Engineering and Applications，2021，57（6）：10-21.
[3] DEVLIN J，CHANG M W，LEE K，et al.BERT：pre-training of deep bidirectional transformers for language understanding[J].arXiv：1810.04805，2018.
[4] SULEA O M，ZAMPIERI M，MALMASI S，et al.Exploring the use of text classification in the legal domain[J].arXiv：1710.09306，2017.
[5] POLPINIJ J，BHEGANAN P，LUAPHOL B，et al.Identifying of decision components in thai civil case decision by text classification technique[C]//International Conference on Computing and Information Technology.Cham：Springer，2020：11-20.
[6] BLUNSOM P，KOCIK K，CURRAN J R.Question classification with log-linear models[C]//Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval，2006：615-616.
[7] JIA B B，ZHANG M L.Multi-dimensional classification via kNN feature augmentation[J].Pattern Recognition，2020，106：107423.
[8] 杨志明，王来奇，王泳.深度学习算法在问句意图分类中的应用研究[J].计算机工程与应用，2019，55（10）：154-160.
YANG Z M，WANG L Q，WANG Y.Application research of deep learning algorithm in question intention classification[J].Computer Engineering and Applications，2019，55（10）：154-160.
[9] KIM Y.Convolutional neural networks for sentence classification[J].arXiv：1408.5882，2014.
[10] MIKOLOV T，CHEN K，CORRADO G，et al.Efficient estimation of word representations in vector space[J].arXiv：1301.3781，2013.
[11] 史梦飞，杨燕，贺樑，等.基于Bi-LSTM和CNN并包含注意力机制的社区问答问句分类方法[J].计算机系统应用，2018，27（9）：157-162.
SHI M F，YANG Y，HE L，et al.Question categorization of community question answering by combining Bi-LSTM and CNN with attention mechanism[J].Computer Systems & Applications，2018，27（9）：157-162.
[12] SHI Y，YAO K，TIAN L，et al.Deep LSTM based feature mapping for query classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2016：1501-1511.
[13] 余珊珊，苏锦钿，李鹏飞.一种基于自注意力的句子情感分类方法[J].计算机科学，2020，47（4）：204-210.
YU S S，SU J X，LI P F.Sentiment classification method for sentence via self-attention[J].Computer Science，2020，47（4）：204-210.
[14] 吴汉瑜，严江，黄少滨，等.用于文本分类的CNN_BiLSTM_
Attention混合模型[J].计算机科学，2020，47（S2）：23-27.
WU H Y，YAN J，HUANG S B，et al.CNN_BiLSTM_
attention hybrid model for text classification[J].Computer Science，2020，47（S2）：23-27.
[15] PENNINGTON J，SOCHER R，MANNING C D.Glove：global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing（EMNLP），2014：1532-1543.
[16] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Neural Information Processing Systems，2017：5998-6008.
[17] BA J L，KIROS J R，HINTON G E.Layer normalization[J].arXiv：1607.06450，2016.
[18] LECUN Y，BENGIO Y，HINTON G.Deep learning[J].Nature，2015，521（7553）：436.
[19] GU J，WANG Z，KUEN J，et al.Recent advances in convolutional neural networks[J].Pattern Recognition，2018，77：354-377.
[20] SZEGEDY C，LIU W，JIA Y，et al.Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2015：1-9.
[21] IOFFE S，SZEGEDY C.Batch normalization：accelerating deep network training by reducing internal covariate shift[C]//International Conference on Machine Learning，2015：448-456.
[22] GLOROT X，BORDES A，BENGIO Y.Deep sparse rectifier neural networks[C]//Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics，2011：315-323.
[23] KINGMA D P，BA J.Adam：a method for stochastic optimization[J].arXiv：1412.6980，2014.
[24] LAI S，XU L，LIU K，et al.Recurrent convolutional neural networks for text classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2015.
[25] LIU P，QIU X，HUANG X.Recurrent neural network for text classification with multi-task learning[J].arXiv：1605.05101，2016.