关键词: 语义相似度, Transformer编码器, 交互注意力机制, 语义表示

Abstract:

The calculation of semantic similarity aims to calculate the similarity between texts at the semantic level, which is an important task in natural language processing. Aiming at the problem that the existing calculation methods cannot fully represent the semantic features of sentences, the model TEAM based on the Transformer encoder is proposed. It can extract the semantic information in sentences by using the contextual semantic encoding ability of the Transformer model. In addition, the interactive attention mechanism is introduced. When encoding two sentences the interactive attention mechanism is used to extract similar features between the two sentences, making the model better at capturing important semantic information within the sentence and improving the model’s understanding of semantics and generalization capabilities. The experimental results show that the model can improve the accuracy of the results on the semantic similarity calculation task of English and Chinese, and exhibit better results than existing methods.

Key words: semantic similarity, Transformer encoder, interactive attention machanism, semantic representation