Adaptive Neighbor Local Ratio Sum Linear Discriminant Analysis

doi:10.3778/j.issn.1002-8331.2204-0348

Abstract

Abstract: In machine learning and pattern recognition, dimension reduction can significantly improve the discriminative performance and efficiency of the classifier. Ratio sum（RS）is a completely new variant of linear discriminant analysis（LDA） that tries to optimize the projection matrix in every dimension. However, the RS does not take into account the local geometry of the data, which may lead to the failure to obtain the optimal solution. To overcome this disadvantage of RS, an adaptive neighbor local ratio sum linear discriminant analysis algorithm is proposed. The algorithm uses the adaptive neighbor composition method to construct the neighbor matrix, retains the local geometric structure of the data to complete the construction between and within the data matrix, so as to better find the optimal representation of the data. And the method adopts the effective kernel-free parameter neighborhood allocation strategy to construct the neighbor matrix, to avoid the need to adjust the heat core parameters. Finally, comparative experiments on the UCI dataset verifies the effectiveness of the algorithm.

Key words: dimensionality reduction, ratio sum（RS）, linear discriminant analysis（LDA）, adaption neighbor comparative

摘要： 在机器学习和模式识别中，降维能够显著提升分类器的判别性能与效率。比率和（ratio sum，RS）是线性判别分析（linear discriminant analysis，LDA）的一种全新变体，它试图使投影矩阵在每个维度上都达到最优。但RS并没有考虑到数据的局部几何结构，这就可能导致无法求得最优解。为了克服RS的这一缺点，提出了一种自适应近邻局部比值和线性判别分析算法（adaptive neighbor local ratio sum linear discriminant analysis，ANLRSLDA）。该算法使用自适应近邻的构图方法构建邻接矩阵，保留数据的局部几何结构完成了数据类间及类内矩阵的构建，从而更好地找到数据的最优表示；并且该方法采用有效的无核参数邻域分配策略来构造邻接矩阵，避免调整热核参数的需要。在UCI数据集及人脸数据集进行了对比实验，验证了该算法的有效性。

关键词: 降维, 比值和, 线性判别分析, 自适应近邻

ZHANG Jiale, LIN Haoshen, ZHOU Keyi, SUN Bo, YANG Xiaojun. Adaptive Neighbor Local Ratio Sum Linear Discriminant Analysis[J]. Computer Engineering and Applications, 2023, 59(15): 115-122.

张家乐, 林浩申, 周科艺, 孙博, 杨晓君. 自适应近邻局部比值和线性判别分析算法[J]. 计算机工程与应用, 2023, 59(15): 115-122.

References

[1] 黄冬梅，梁素玲，王振华，等.利用信息熵的高光谱遥感影像降维方法[J].计算机工程与应用，2019，55（6）：191-196.
HUANG D M，LIANG S L，WANG Z H，et al.Dimensionality reduction method for hyperspectral remote sensing image based on information entropy[J].Computer Engineering and Applications，2019，55（6）：191-196.
[2] WOLD S，ESBENSEN K，GELADI P.Principal component analysis[J].Chemometrics and Intelligent Laboratory Systems，1987，2（1/3）：37-52.
[3] LI B N，YU Q，WANG R，et al.Block principal component analysis with nongreedy[?1]-norm maximization[J].IEEE Transactions on Cybernetics，2015，46（11）：2543-2547.
[4] BELHUMEUR P N，HESPANHA J P，KRIEGMAN D J.Eigenfaces vs.fisherfaces：recognition using class specific linear projection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，1997，19（7）：711-720.
[5] LI H F，JIANG T，ZHANG K S.Efficient and robust feature extraction by maximum margin criterion[J].IEEE Transactions on Neural Networks，2003，17（1）：157-165.
[6] SUGIYAMA M.Local fisher discriminant analysis for supervised dimensionality reduction[C]//Proceedings of the 23rd International Conference on Machine Learning，2006：905-912.
[7] HE X，NIYOGI P.Locality preserving projections[C]//Conference on Advances in Neural Information Processing Systems，2003.
[8] CAI D，HE X，ZHOU K，et al.Locality sensitive discriminant analysis[C]//Proceedings of the 20th International Joint Conference on Artificial Intelligence，2007：1713-1726.
[9] NIE F，WANG Z，WANG R，et al.Adaptive local linear discriminant analysis[J].ACM Transactions on Knowledge Discovery from Data（TKDD），2020，14（1）：1-19.
[10] WANG Z，NIE F，WANG R，et al.Local structured feature learning with dynamic maximum entropy graph[J].Pattern Recognition，2021，111：107673.
[11] WANG J，WANG H，NIE F，et al.Ratio sum vs.sum ratio for linear discriminant analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2022，44（12）：10171-10185.
[12] LIANG K，YANG X J，XU Y X，et al.Ratio sum formula for dimensionality reduction[J].Multimedia Tools and Applications，2021，80（3）：4367-4382.
[13] NIE F，WANG X，HUANG H，et al.Clustering and projected clustering with adaptive neighbors[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining，2014：977-986.
[14] 梁珂.基于Ratio sum的子空间降维算法研究与应用[D].广州：广东工业大学，2021.
LIANG K.Thesearch and application of subspace dimensionalityreduction algorithm based on Ratio sum problem[D].Guangzhou：Guangdong University of Technology，2021.
[15] 刘云东，李鸿，白万荣，等.一种自适应邻域选择半监督判别分析算法[J].计算机工程与应用，2011，47（35）：180-183.
LIU Y D，LI H，BAI W R，et al.Semi-supervised discriminant analysis algorithm based on adaptive neighborhoodselection[J].Computer Engineering and Applications，2011，47（35）：180-183.