Oversampling Algorithm Based on Density Peak Clustering and Radial Basis Function

doi:10.3778/j.issn.1002-8331.2103-0564

Abstract

Abstract: Most of the existing oversampling algorithms only consider the distribution of the minority instances but ignore the distribution of the majority instances in the sampling process. In addition to the problem of imbalance between classes, the data set also has the problem of imbalance within classes. To solve these problems, this paper proposes a new oversampling method based on density peak clustering and radial basis function. Firstly, the minority instances are adaptively clustered by the improved density peak clustering algorithm, and a number of minority sub-clusters are obtained. Secondly, the local density calculated by the clustering process is used to assign weights to each sub-cluster, which are used to determine the required number of each sub-cluster. Finally, the radial basis function is used to calculate the mutual minority class potential of each minority instances, and the minority class is oversampled based on the mutual minority class potential. The proposed algorithm is combined with different classifiers to conduct experiments, and different indicators are used to evaluate the performance. The experiment shows that the performance of the proposed algorithm is better.

Key words: imbalanced data, oversampling, density peak clustering, radial basis function

摘要： 现有的大多数过采样算法在采样过程中只考虑少数类样本的分布而忽略多数类样本的分布，且数据集除了存在类间不平衡问题之外，还存在类内不平衡问题。针对这些问题，提出一种基于密度峰值聚类和径向基函数的过采样方法。该方法首先利用改进的密度峰值聚类算法自适应地为少数类聚类，获得多个子簇；利用聚类过程计算所得的局部密度为各子簇分配权重，并根据权重确定各子簇的过采样量；用径向基函数计算少数类样本的相互类势，以相互类势为依据对少数类进行过采样。将算法与不同分类器结合进行实验，用不同指标评价分类效果，实验表明，该算法的分类效果较优。

关键词: 不平衡数据, 过采样, 密度峰值聚类, 径向基函数

LU Miaofang, YANG Youlong. Oversampling Algorithm Based on Density Peak Clustering and Radial Basis Function[J]. Computer Engineering and Applications, 2022, 58(21): 67-74.

陆妙芳, 杨有龙. 基于密度峰值聚类和径向基函数的过采样算法[J]. 计算机工程与应用, 2022, 58(21): 67-74.

References

[1] TAO X，LI Q，GUO W，et al.Self-adaptive cost weights-based support vector machine cost-sensitive ensemble for imbalanced data classification[J].Information Sciences，2019，487：31-56.
[2] TSAI C F，LIN W C，HU Y H，et al.Under-sampling class imbalanced datasets by combining clustering analysis and instance selection[J].Information Sciences，2019，477：47-54.
[3] HASSAN M M，HUDA S，YEARWOOD J，et al.Multistage fusion approaches based on a generative model and multivariate exponentially weighted moving average for diagnosis of cardiovascular autonomic nerve dysfunction[J].Information Sciences，2018，41：105-118.
[4] FIORE U，DE SANTIS A，PERLA F，et al.Using generative adversarial networks for improving classification effectiveness in credit card fraud detection[J].Information Sciences，2019，479：448-455.
[5] TAN X P，SU S J，HUANG Z P，et al.Wireless sensor networks intrusion detection based on SMOTE and the random forest algorithm[J].Sensors，2019，19（1）：203.
[6] LI Y J，GUO H X，ZHANG Q P，et al.Imbalanced text sentiment classification using universal and domain-specific knowledge[J].Knowledge-Based Systems，2018，160：1-15.
[7] TAO X M，LI Q，REN C，et al.Real-value negative selection over-sampling for imbalanced dataset learning[J].Expert Systems with Applications，2019，129：118-134
[8] CHAWLA N V，BOWYER K W，HALL L O，et al.SMOTE：synthetic minority over-sampling technique[J].Journal of Artificial Intelligence Research，2002，16：321-357.
[9] BUNKHUMPORNPAT C，SINAPIROMSARAN K，LURSINSAP C.Safe-Level-SMOTE：safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem[C]//Proceedings of the 13th Pacific-Asia Conference on Advances in Knowledge Discovery & Data Mining，Bangkok，April 27-30，2009.Berlin，Germany：Springer-Verlag，2009：475-482.
[10] HAN H，WANG W Y，MAO B H.Borderline-SMOTE：a new over-sampling method in imbalanced data sets learning[C]//Proceedings of the Advances in Intelligent Computing，2005：878-887.
[11] YANG X，KUANG Q，ZHANG W，et al.AMDO：an over-sampling technique for multi-class imbalanced problems[J].IEEE Transactions on Knowledge & Data Engineering，2018（9）：1672-1685.
[12] MAHALANOBIS P C.On the generalised distance in statistics[J].Proceedings of the National Institute of Sciences of India，1936，2：49-55.
[13] CIESLAK D A，CHAWLA N V，STRIEGEL A.Combating imbalance in network intrusion datasets[C]//IEEE International Conference on Granular Computing，Atlanta Georgia，May 10-12，2006.USA：IEEE，2006：732-737.
[14] BUNKHUMPORNPAT C，SINAPIROMSARAN K，LURSINSAP C.DBSMOTE：density-based synthetic minority oversampling technique[J].Applied Intelligence，2012，36（3）：664-684.
[15] ANAND A，PUGALENTHI G，FOGEL G B，et al.An approach for classification of highly imbalanced data using weighting and undersampling[J].Amino Acids，2010，39（5）：1385-1391.
[16] KUBAT M，MATWIN S.Addressing the curse of imbalanced training sets：one sided selection[C]//Fourteeth International Conference on Machine Learning，2000：179-186.
[17] YEN S J，LEE Y S.Cluster-based under-sampling approaches for imbalanced data distributions[J].Expert Systems with Applications，2009，36（3）：5718-5727.
[18] SONG J，HUANG X L.A bi-directional sampling based on k-means method for imbalance text classification[C]//IEEE/ACIS 15th International Conference on Computer and Information Science（ICIS），Okayama Japan，June 26-29，2016.USA：IEEE，2016：1-5.
[19] SIERS M J，ISLAM M Z.Novel algorithms for cost-sensitive classification and knowledge discovery in class imbalanced datasets with an application to NASA software defects[J].Information Sciences，2018，459：53-70.
[20] CHAWLA N V，LAZAREVIC A，HALL L O，et al.SMOTEBoost：improving prediction of the minority class in boosting[C]//European Conference on Principles of Data Mining and Knowledge Discovery，2003：107-119.
[21] SEIFFERT C，KHOSHGOFTAAR T M，VAN HULSE J，et al.RUSBoost：a hybrid approach to alleviating class imbalance[J].IEEE Transactions on Systems Man and Cybernetics-Part A Systems and Humans，2010，40（1）：185-197.
[22] SUN J，LANG J，FUJITA H，et al.Imbalanced enterprise credit evaluation with DTE-SBD：decision tree ensemble based on SMOTE and bagging with differentiated sampling rates[J].Information Sciences，2018，425：76-91.
[23] RODRIGUEZ A，LAIO A.Clustering by fast search and find of density peaks[J].Science，2014，344：1492-1496.
[24] PARMAR M，WANG D，ZHANG X F，et al.REDPC：a residual error-based density peak clustering algorithm[J].Neurocomputing，2019，348：82-96.
[25] ZELNIK-MANO L，PERONA P.Self-tuning spectral clustering[C]//Proceedings of the 17th International Conference on Neural Information Processing Systems，2005：1601-1608.
[26] NEKOOEIMEHR I，LAI-YUEN S K.Adaptive semi-unsupervised weighted oversampling（A-SUWO） for imbalanced datasets[J].Expert Systems with Applications，2016，46（5）：405-416.
[27] DOUZAS G，BACAO F，LAST F.Improving imbalanced learning through a heuristic oversampling method based on k-means and SMOTE[J].Information Sciences，2018，465：1-20.