关键词: 多标签文本分类, 多头注意力机制, 胶囊网络, 双向长短期记忆网络, 联合模型

Abstract:

At present, the multi-label text classification algorithm ignores the importance of different words in text sequences and the influence of different levels of text features. This paper proposes an ATT-Capsule-BiLSTM method based on multi-head attention, CapsuleNet and the Bidirectional Long Short-Term Memory network（BiLSTM） model. Firstly, the text sequence is vectorized, and the weight distribution of the words is learned by multi-head attention on the basis of the word vector. Then the feature representation of the local spatial information and the context timing information are extracted by the Capsule network and BiLSTM respectively, and the fusion is performed through the fusion layer. After that, it is classified by the sigmoid classifier. The comparison experiments are carried out on two data sets, Reuters-21578 and AAPD. The experimental results show that the proposed joint model achieves better performance based on simple architecture. The [F1] values reach 89.82% and 67.48% respectively.

Key words: multi-label text classification, multi-head attention, CapsuleNet, Bidirectional Long Short-Term Memory network（BiLSTM）, joint model