Computer Engineering and Applications ›› 2020, Vol. 56 ›› Issue (11): 17-25.DOI: 10.3778/j.issn.1002-8331.2001-0285

Previous Articles     Next Articles

Review of Machine Reading Comprehension Based on Pre-training Language Model

ZHANG Chaoran, QIU Hangping, SUN Yi, WANG Zhongwei   

  1. 1.College of Command & Control Engineering, Army Engineering University of PLA, Nanjing 210007, China
    2.Unit 73658 of PLA, China
  • Online:2020-06-01 Published:2020-06-01

基于预训练模型的机器阅读理解研究综述

张超然,裘杭萍,孙毅,王中伟   

  1. 1.陆军工程大学 指挥控制工程学院,南京 210007
    2.中国人民解放军73658部队

Abstract:

In recent years, deep learning technology has been advancing. With the application and development of pre-training model in natural language processing, machine reading comprehension is no longer simply based on the combination of network structure and word embedding. The development of pre-training language model has led to advances in machine reading comprehension that has surpassed human performance in some datasets. This paper briefly introduces the concepts of machine reading comprehension and pre-training language model, summarizes the current research progress of machine reading comprehension based on the pre-training model, analyzes the performance of the current pre-training model on the relevant data set, summarizes the existing problems and looks forward to the future.

Key words: deep learning, pre-training model, natural language processing, machine reading comprehension

摘要:

近年来深度学习技术不断进步,随着预训练模型在自然语言处理中的应用与发展,机器阅读理解不再单纯地依靠网络结构与词嵌入相结合的方法。预训练语言模型的发展推动了机器阅读理解的进步,在某些数据集上已经超越了人类的表现。简要介绍机器阅读理解以及预训练语言模型的相关概念,综述当下基于预训练模型的机器阅读理解研究进展,对目前预训练模型在相关数据集上的性能进行分析,总结了目前存在的问题并对未来进行展望。

关键词: 深度学习, 预训练模型, 自然语言处理, 机器阅读理解