Abstract: Domain adaptation, the important branch of transfer learning, aims to solve the problem that the performance of traditional machine learning algorithms drops sharply when the training and test samples obey different data distributions. Transformer is a deep learning framework based on a self-attention mechanism, which has strong global feature extraction ability and modeling ability. In recent years, the combination of Transformer and domain adaptation has also become a research hotspot. Although many relative methods have been published, the review of Transformer application in domain adaptation has not been reported. In order to fill the gap in this field and provide reference for relevant research, this paper summarizes and analyzes some typical domain adaptation methods based on Transformer in recent years. This paper summarizes the concepts related to domain adaptation and the basic structure of the Transformer, sorts out various domain adaptation methods based on Transformer from four applications, i.e. image classification, image semantic segmentation, object detection and medical image analysis and compares the domain adaptation methods in image classification. Finally, the challenges of the current domain adaptation Transformer model are summarized, and the feasible research directions in the future are discussed.

Key words: domain adaptation, transfer learning, Transformer, self-attention mechanism, deep learning

摘要： 作为迁移学习的重要分支，域适应旨在解决传统机器学习算法在训练样本和测试样本服从不同数据分布时性能急剧下降的问题。Transformer是基于自注意力机制的深度学习框架，具有强大的全局特征提取能力和建模能力，近年来Transformer与域适应相结合也成为研究的热点。虽然已有大量相关方法问世，但Transformer应用在域适应的综述却未见报道。为了填补这个领域的空白，为相关研究提供借鉴和参考，对近年来出现的一些基于Transformer的典型域适应方法进行归纳总结与分析，概述域适应的相关概念与Transformer的基本结构，从图像分类、图像语义分割、目标检测、医学图像分析四个应用梳理了各种基于Transformer的域适应方法，对图像分类下的域适应方法进行比较，总结当前域适应Transformer模型存在的挑战并探讨未来可行的研究方向。

关键词: 域适应, 迁移学习, Transformer, 自注意力机制, 深度学习