关键词: 双向长短期记忆网络, 条件随机场, 自注意力机制, 语义角色标注, 深度学习

Abstract:

A SelfAtt-BILSTM-CRF model is constructed by combining the Self Attention Mechanism with Bidirectional Long Short-Term Memory（SelfAtt-BILSTM） and Conditional Random Field（CRF）. The semantic role of policy text is annotated to extract the main content of policy. This paper takes a college policy file as the experimental data set, uses the BILSTM model to automatically learn the context features of serialized statements, integrates the self attention mechanism to increase the weight of important feature elements, and finally uses the features to sequence annotation through CRF layer to extract the semantic roles, so as to realize the main content mining of policy files. By contrast, the SelfAtt-BILSTM-CRF model can effectively extract the content of policy text, and the F1 value in the annotation data set reaches 78.99%. The experimental results also show that the Self Attention Mechanism can effectively improve the semantic role annotation effect of the neural network model.

Key words: bidirectional long short-term memory, conditional random field, self attention mechanism, semantic role labeling, deep learning