Domain Adaptation Method Combined with Discriminant Analysis and Distribution Discrepancy Constraints

doi:10.3778/j.issn.1002-8331.2303-0035

Abstract

Abstract: To solve the problem of category structure loss and local feature loss caused by feature transformation based on global distribution adaptation in the process of domain adaptation, a domain adaptation method combined with discriminant analysis and distribution discrepancy constraints is proposed. Firstly, the mean distance metric of the domain data distribution is constructed for domain distribution adaptation; secondly, the class divergence metric is constructed to maintain the class discriminant structure; meanwhile, based on the sample local distribution information, multiple discrepancy weights are designed to constrain the domain distribution discrepancy measure and the class discrimination measure respectively, which realizes the joint optimization of discriminant preservation and locality preservation; finally, based on the feature transformation optimized by the metrics aforementioned, the source domain data and target domain data are projected into the subspace for classification task. During the process of domain adaptation, the proposed method can not only reduce the distribution difference between domains, but also take into account the preservation of category discrimination and local characteristics of data, thus effectively improve the performance of reusage of out-of-domain data. The experimental results on 28 cross-domain classification tasks show that the proposed method significantly outperforms the other related methods in terms of the evaluation metric.

Key words: domain adaptation, inner-class scatter, intra-class scatter, discriminant analysis, distribution discrepancy

摘要： 为解决领域适应过程中基于全局分布适配的特征变换造成的类别结构损失和局部特征损失问题，提出一种结合判别分析和分布差异约束的领域适应方法。构造领域数据分布均值距离度量用于领域间分布适配；构造类散度度量用于保持类别判别结构；基于数据局部分布信息设计不同类型的差异权重，分别用于约束域分布距离度量和类散度度量，实现判别保持和局部保持的联合优化；基于上述度量最优化的特征变换，将源域和目标域数据投影到子空间中实施分类任务。所提出的方法在领域适应过程中不仅能够缩小领域间分布差异，且兼顾类别判别保持和数据局部特征的保持，能有效提升域外数据重用的性能。在28组跨领域分类任务上的实验结果表明，所提出的方法在评价指标上优于已有的相关方法。

关键词: 领域适应, 类内散度, 类间散度, 判别分析, 分布差异

QIN Jiangwei, TANG Deyu. Domain Adaptation Method Combined with Discriminant Analysis and Distribution Discrepancy Constraints[J]. Computer Engineering and Applications, 2024, 60(2): 77-86.

覃姜维, 唐德玉. 结合判别分析和分布差异约束的领域适应方法[J]. 计算机工程与应用, 2024, 60(2): 77-86.

References

[1] 庄福振, 罗平, 何清, 等. 迁移学习研究进展[J]. 软件学报, 2015, 26(1): 26-39.
ZHUANG F Z, LUO P, HE Q, et al. Survey on transfer learning research[J]. Journal of Software, 2015, 26(1): 26-39.
[2] WILSON G, COOK D J. A survey of unsupervised deep domain adaptation[J]. ACM Transactions on Intelligent Systems and Technology, 2020, 11(5): 1-46.
[3] GRETTON A, BORGWARDT K M, RASCH M, et al. A kernel method for the two sample problem[C]//Proceedings of the 19th International Conference on Neural Information Processing Systems, Canada, December 4-7, 2006. Cambridge, MA, USA: MIT Press, 2006: 513-520.
[4] SHEN J, QU Y, ZHANG W, et al. Wasserstein distance guided representation learning for domain adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2018: 4058-4065.
[5] LI J J, CHEN E P, DING Z M, et al. Maximum density divergence for domain adaptation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(11): 3918-3930.
[6] ZHOU G Y, HUANG J X. Modeling and mining domain shared knowledge for sentiment analysis[J]. ACM Transactions on Information Systems, 2018, 36(2): 1-36.
[7] WANG J, FENG W, CHEN Y, et al. Visual domain adaptation with manifold embedded distribution alignment[C]//Proceedings of the 26th ACM International Conference on Multimedia, Seoul Republic of Korea, October 22-26, 2018. New York, NY, United States: ACM, 2018: 402-410.
[8] DENG C, LIU X, LI C, et al. Active multi-kernel domain adaptation for hyperspectral image classification[J]. Pattern Recognition, 2018, 77: 306-315.
[9] LIU W, QIN R. A multikernel domain adaptation method for unsupervised transfer learning on cross-source and cross-region remote sensing data classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(6): 4279-4289.
[10] LONG M, ZHU H, WANG J, et al. Deep transfer learning with joint adaptation networks[C]//Proceedings of the 34th International Conference on Machine Learning, Sydney, August 6-11, 2017. Cambridge, MA, USA: MIT Press, 2017: 2208-2217.
[11] ZHANG Z, LIU Y, HAN C, et al. Generalized one-shot domain adaptation of generative adversarial networks[J]. arXiv:2209.
03665, 2022.
[12] MA X, ZHANG T, XU C. GCAN: graph convolutional adversarial network for unsupervised domain adaptation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, June 15-20, 2019. Piscataway: IEEE, 2019: 8258-8268.
[13] GAO Z, ZHANG S, HUANG K, et al. Gradient distribution alignment certificates better adversarial domain adaptation[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, October 10-17, 2021. Piscataway: IEEE, 2021: 8917-8926.
[14] LI S, SONG S, HUANG G, et al. Domain invariant and class discriminative feature learning for visual domain adaptation[J]. IEEE Transactions on Image Processing, 2018, 27(9): 4260-4273.
[15] DENG W, LIAO Q, ZHAO L, et al. Joint clustering and discriminative feature alignment for unsupervised domain adaptation[J]. IEEE Transactions on Image Processing, 2021, 30: 7842-7855.
[16] LI S, LIU C H, SU L, et al. Discriminative transfer feature and label consistency for cross-domain image classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(11): 4842-4856.
[17] WANG W, SHEN Z, LI D, et al. Probability-based graph embedding cross-domain and class discriminative feature learning for domain adaptation[J]. IEEE Transactions on Image Processing, 2023, 32: 72-87.
[18] HUANG Y, PENG J, NING Y, et al. Graph embedding and distribution alignment for domain adaptation in hyperspectral image classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 7654-7666.
[19] LI J, JING M, LU K, et al. Locality preserving joint transfer for domain adaptation[J]. IEEE Transactions on Image Processing, 2019, 28(12): 6103-6115.
[20] SANODIYA R K, MISHRA S, SINGH R S R, et al. Manifold embedded joint geometrical and statistical alignment for visual domain adaptation[J]. Knowledge-Based Systems, 2022, 257: 109886.
[21] WU H, NG M K. Multiple graphs and low-rank embedding for multi-source heterogeneous domain adaptation[J]. ACM Transactions on Knowledge Discovery from Data, 2022, 16(4): 1-25.
[22] ZHU R, JIANG X, LU J, et al. Transferable feature learning on graphs across visual domains[C]//2021 IEEE International Conference on Multimedia and Expo (ICME), Shenzhen, July 5-9, 2021. Piscataway: IEEE, 2021: 1-6.
[23] ZHANG J, LI W, OGUNBONA P. Joint geometrical and statistical alignment for visual domain adaptation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, July 21-26, 2017. Piscataway: IEEE, 2017: 5150-5158.
[24] CHENG Z. Robust and high-order correlation alignment for unsupervised domain adaptation[J]. Neural Computing and Applications, 2021, 33(12): 6891-6903.
[25] HUANG Z, WEN J, CHEN S, et al. Discriminative radial domain adaptation[J]. arXiv:2301.00383, 2023.
[26] LONG M, WANG J, DING G, et al. Transfer joint matching for unsupervised domain adaptation[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, June 23-28, 2014, Piscataway: IEEE, 2014: 1410-1417.
[27] ZHU Y, ZHUANG F, WANG J, et al. Multi-representation adaptation network for cross-domain image classification[J]. Neural Networks, 2019, 119: 214-221.
[28] LONG M, WANG J, DING G, et al. Transfer feature learning with joint distribution adaptation[C]//2013 IEEE International Conference on Computer Vision, Australia, December 1-8, 2013. Piscataway: IEEE, 2013: 2200-2207.
[29] ZHAO D, LIN Z, XIAO R, et al. Linear Laplacian discrimination for feature extraction[C]//2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, June 17-22, 2007. Piscataway: IEEE, 2007: 1-7.
[30] PAN S J, TSANG I W, KWOK J T, et al. Domain adaptation via transfer component analysis[J]. IEEE Transactions on Neural Networks, 2011, 22(2): 199-210.
[31] GONG B Q, SHI Y, SHA F, et al. Geodesic flow kernel for unsupervised domain adaptation[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition, RI, June 16-21, 2012. Washington: IEEE Computer Society, 2012: 2066-2073.
[32] SUN B, FENG J, SAENKO K. Return of frustratingly easy domain adaptation[C]//Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. Phoenix, Arizona: AAAI Press, 2016: 2058-2065.
[33] XIAO T, LIU P, ZHAO W, et al. Structure preservation and distribution alignment in discriminative transfer subspace learning[J]. Neurocomputing, 2019, 337: 218-234.