Marginal Manifold Embedding for Feature Extraction

doi:10.3778/j.issn.1002-8331.2105-0450

Abstract

Abstract: Marginal information of sample points is of great importance for classification. In view of fact that marginal information of sample points used in constructing the intrinsic graph and the penalty graph by marginal Fisher analysis（MFA） and locality sensitive discriminant analysis（LSDA） can’t sufficiently characterize the separability of different classes of sample points in some cases, a novel graph embedding dimensionality reduction algorithm called marginal manifold embedding（MME） is proposed. According to the label information of sample points, firstly, MME algorithm finds the nearest heterogeneous margin sub-manifold of each sample point, then returns to its own category to find corresponding homogenous margin sub-manifold that is the nearest one to the heterogeneous margin sub-manifold, thus defining the homogeneous margin neighbourhood and the heterogeneous margin neighbourhood closely related to different classes of sample points. Finally, by maximizing the distance between all paired margin sub-manifolds, MME algorithm can obtain a low-dimensional space with more discriminant significance. At the same time, MME algorithm can trap outliers wandering in the margin into the marginal neighbourhood, which is helpful to reduce the negative influence of outliers on the algorithm. Experimental results on face datasets show that the low-dimensional features extracted by MME algorithm can improve the classification accuracy.

Key words: margin, classification, graph embedding, dimensionality reduction, outliers

摘要： 样本点的边界信息对于分类具有重要意义。针对于边界Fisher分析（MFA）和局部敏感判别分析（LSDA）构造本征图和惩罚图所利用的样本点边界信息，在一些情况下并不能很好地表征不同类样本点的可分性，提出了一种新的图嵌入降维算法——边界流形嵌入（MME）。MME算法根据样本点的标签信息，寻找距离每个样本点最近的异类边界子流形，再返回本类中寻找距离异类边界子流形最近的同类边界子流形，从而定义出不同类样本间密切联系的同类边界邻域和异类边界邻域。通过最大化所有成对的边界子流形之间的距离，MME算法可以得到更具有鉴别意义的低维特征空间。同时，MME算法能将徘徊在边界的离群点收入到边界邻域里，这对减弱离群点给算法带来的负面的影响有一定的帮助。在人脸数据库上的实验结果表明了MME算法提取的低维特征能够提升分类的准确率。

关键词: 边界, 分类, 图嵌入, 降维, 离群点

GONG Sicong, XU Jie, WAN Minghua. Marginal Manifold Embedding for Feature Extraction[J]. Computer Engineering and Applications, 2022, 58(23): 245-253.

龚思聪, 徐洁, 万鸣华. 边界流形嵌入在特征提取中的应用[J]. 计算机工程与应用, 2022, 58(23): 245-253.

References

[1] YI Y G，WANG J Z，ZHOU W，et al.Joint graph optimization and projection learning for dimensionality reduction[J].Pattern Recognition，2019，92：258-273.
[2] JIN C，JIN S W，QIN L N.Attribute selection method based on a hybrid BPNN and PSO algorithm[J].Applied Soft Computing，2012，12（8）：2147-2155.
[3] NIE R C，ZHOU D M，HE M，et al.Facial feature extraction using frequency map series in PCNN[J].Journal of Sensors，2016：1-9.
[4] YU C B，NIE R C，ZHOU D M.A regularized locality projection-based sparsity discriminant analysis for face recognition[J].International Journal of Pattern Recognition and Artificial Intelligence，2018，32（5）：1856006.
[5] ABUDARHAM N，SHKILLER L，YOVEL G.Critical features for face recognition[J].Cognition，2019，182：73-83.
[6] GUI J，JIA W，ZHU L，et al.Locality preserving discriminant projections for face and palmprint recognition[J].Neurocomputing，2010，73（13）：2696-2707.
[7] CHEN B，ZHANG H，ZHANG X F，et al.Max-margin discriminant projection via data augmentation[J].IEEE Transactions on Knowledge and Data Engineering，2015，27（7）：1964-1976.
[8] YI Y G，WANG J Z，ZHOU W，et al.Non-negative matrix factorization with locality constrained adaptive graph[J].IEEE Transactions on Circuits and Systems for Video，2019，30（2）：427-441.
[9] JIN C，JIN S W.Automatic image annotation using feature selection based on improving quantum particle swarm optimization[J].Signal Processing，2015，109：172-181.
[10] JING X Y，LAN C，ZHANG D，et al.Face feature extraction and recognition based on discrimint subclass-center manifold preserving projection[J].Pattern Recognition Letters，2012，33（6）：709-717.
[11] YI Y G，QIAO S L，ZHOU W X，et al.Adaptive multiple graph regularized semi-supervised extreme learning machine[J].Soft Computing，2018，22（11）：3545-3562.
[12] TURK M，PENTLAND A.Face recognition using eigenfaces[C]//Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition，1991：586-591.
[13] TURK M，PENTLAND A.Eigenfaces for recognition[J].Journal of Congitive Neuroscience，1991，3（1）：71-86.
[14] WEI L，ZHOU R G，YIN J，et al.Latent graph-regularized inductive robust principal componentanalysis[J].Knowledge-Based Systems，2019，177：68-81.
[15] BELHUMEUR P N，HESPANHA J P，KRIEGMAN D J.Eigenfaces vs. fisherfaces：recognition using class specific linear projection[J].IEEE Transcations on Pattern Analysis and Machine Intelligence，1997，19（7）：711-720.
[16] QU D P，TAN K，DU Q.Feature extraction for hyperspectral images based on semi-supervised local discrimi-nant analysis[J].Remote Sensing，2019，11（7）：654-671.
[17] CHOI H，CHOI S.Robust kernel isomap[J].Pattern Recognition，2007，40（3）：853-862.
[18] ZHANG Y，ZHANG Z，JIE Q，et al.Semi-supervised local multi-manifold Isomap by linear embedding for feature extraction[J].Pattern Recognition，2018，76：662-678.
[19] ROWEIS S T，SAUL L K.Nonlinear dimensionality reduction by locally linear embedding[J].Science，2000，290（5500）：2323-2326.
[20] SHI G Y，HUANG H，LIU J M，et al.Spatial-spectral multiple manifold discriminant analysis for dimensionality reduction of hyperspectral imagery[J].Remote Sensing，2019，11（20）：2414-2437.
[21] BELKIN M，NIYOGI P.Laplacian eigenmaps for dimensionality reduction and data representation[J].Neural Computation，2003，15（6）：1373-1396.
[22] SHI G Y，HUANG H，WANG L H.Unsupervised dimensionality reduction for hyperspectral imagery via local geometric structure feature learning[J].IEEE Geoscience and Remote Sensing Letters，2020，17（8）：1425-1429.
[23] HE X F，CAI D，YAN S C，et al.Neighborhood preserving embedding[C]//Tench IEEE International Conference on Computer Vision，2005：1208-1213.
[24] CHEN H T，WU J P，JIANG B，et al.A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber-physical systems[J].ISA Transactions，2020，104：175-183.
[25] HE X F，YAN S C，HU Y X，et al.Face recognition using Laplacianfaces[J].IEEE Transcations on Pattern Analysis and Machine Intelligence，2005，27（3）：328-340.
[26] GAO Y L，ZHONG S X，HU K L，et al.Robust locality preserving projections using angle-based adaptive weight method[J].IET Computer Vision，2020，14（8）：605-613.
[27] 薛啸宇，马小虎.基于双邻接图正交近邻保持投影的人脸识别算法[J].计算机科学，2017，44（8）：31-35.
XUE X Y，MA X H.Double adjacency graphs based orthogonal neighborhood preserving projections for face recognition[J].Computer Science，2017，44（8）：31-35.
[28] YAN S C，XU D，ZHANG B Y，et al.Graph embedding and extensions：a general framework for dimensionality reduction[J].IEEE Transcations on Pattern Analysis and Machine Intelligence，2007，29（1）：40-51.
[29] HUANG Z Y，ZHU H Y，ZHOU J T，et al.Multiple marginal fisher analysis[J].IEEE Transactions on Industrial Electronics，2019，66（12）：9798-9807.
[30] CAI D，HE X F，ZHOU K，et al.Locality sensitive discriminant analysis[C]//Proceedings of the 20th International Joint Conference on Artifical Intelligence，2007：708-713.
[31] WU F，JING X Y，HUANG Q H.Uncorrelated locality-sensitive multi-view discriminant analysis[J].National Academy Science Letters，2020，43（4）：327-331.
[32] YANG Z J，HUANG P，WAN M H，et al.Discriminant maximum margin projections for face recognition[J].Multimedia Tools and Applications，2019，78（17）：23847-23865.
[33] YI Y G，ZHANG B X，KONG J，et al.An improved locality sensitive discriminant analysis approach for feature extraction[J].Multimedia Tools and Applications，2015，74（1）：85-104.
[34] WEI Y K，JIN C.Locality sensitive discriminant projection for feature extraction and face recognition[J].Journal of Electronic Imaging，2019，28（4）：1-13.