Computer Engineering and Applications ›› 2023, Vol. 59 ›› Issue (2): 12-21.DOI: 10.3778/j.issn.1002-8331.2209-0025

• Research Hotspots and Reviews • Previous Articles     Next Articles

Review of Research on Adapter and Prompt Tuning

LIN Lingde, LIU Na, WANG Zheng'an   

  1. 1.College of Computer Science and Engineering, North Minzu University, Yinchuan 750021, China
    2.The Key Laboratory of Images & Graphics Intelligent Processing of State Ethnic Affairs Commission, North Minzu University, Yinchuan 750021, China
  • Online:2023-01-15 Published:2023-01-15

Adapter与Prompt Tuning微调方法研究综述


  1. 1.北方民族大学 计算机科学与工程学院,银川 750021
    2.北方民族大学 图像图形智能处理国家民委重点实验室,银川 750021

Abstract: Text mining is a branch of data mining, covering a variety of technologies, among which natural language processing technology is one of the core tools of text mining, which aims to help users obtain useful information from massive data. In recent years, the pre-training model has played an important role in promoting the research and development of natural language processing, and the fine-tuning method of the pre-training model has also become an important research field. On the basis of the relevant literature on the pre-training model fine-tuning method published in recent years, this paper reviews the current mainstream Adapter and Prompt methods. First of all, the development of natural language processing is briefly combed, and the problems and difficulties in fine-tuning of pre-training models are analyzed. Secondly, two kinds of fine-tuning methods:Adapter and Prompt, and the classic methods in the this two research directions are introduced. The advantages, disadvantages and performance are analyzed and summarized. Finally, this paper summarizes the limitations of the current fine-tuning methods of the pre-training model and discusses the future development direction.

Key words: text mining, natural language processing, deep learning, pre-trained models, fine-tuning methods

摘要: 文本挖掘是数据挖掘的一个分支学科,涵盖多种技术,其中自然语言处理技术是文本挖掘的核心工具之一,旨在帮助用户从海量数据中获取有用的信息。近年来,预训练模型对自然语言处理的研究和发展有重要的推动作用,预训练模型的微调方法也成为重要的研究领域。根据近年来预训练模型微调方法的相关文献,选择目前主流的Adapter与Prompt微调方法进行介绍。对自然语言处理的发展脉络进行简要梳理,分析目前预训练模型微调存在的问题与不足;介绍Adapter与Prompt两类微调方法,对两个研究方向中经典方法进行介绍,并从优缺点和性能等方面进行详细分析;进行总结归纳,阐述目前预训练模型的微调方法存在的局限性并讨论未来发展方向。

关键词: 文本挖掘, 自然语言处理, 深度学习, 预训练模型, 微调方法