Multi-classification Ensemble Algorithm for Imbalanced Data

doi:10.3778/j.issn.1002-8331.2008-0058

Abstract

Abstract: To solve the problem of imbalanced multi-classification, an ensemble classification algorithm（IDESF） based on sampling and feature selection is proposed. The diversity of base classifier affects the classification performance of the ensemble algorithm. Therefore, the IDESF algorithm first performs a two-stage sampling of the data set with replacement sampling+SMOTE. On the basis of ensuring the rationality of the samples in the obtained data set, the two-stage sampling increases the difference among the data sets so as to improve the diversity of the base classifier implicitly. In addition, two-stage sampling can also balance the data distribution and prevent the classifier from favoring the majority classes. Based on two-stage sampling, IDESF algorithm introduces data cleaning and feature selection methods to further improve the classification performance of the algorithm. Finally, the results show that the algorithm can obtain higher AUCarea and G-mean and has better classification effect.

Key words: imbalanced data, over-sampling, data cleaning, feature selection, ensemble algorithm, multi-classification

摘要： 为解决不均衡多分类问题，提出了一种基于采样和特征选择的不均衡数据集成分类算法（IDESF）。基分类器的多样性会影响集成算法的分类性能，所以IDESF算法对数据集进行有放回采样+SMOTE的两阶段采样。两阶段采样在保证所得数据集中样本合理性的基础上，增加数据集间的差异性以此隐式地提高基分类器的多样性。两阶段采样同样可以平衡数据分布，防止分类器偏向多数类。在两阶段采样的基础上，IDESF算法引入了数据清洗和特征选择方法，试图进一步提高算法的分类性能。与其他不均衡分类算法在5组不均衡数据集上进行了对比实验，结果表明该算法可以获得较高的AUCarea和G-Mean值，具有较为优异的分类效果。

关键词: 不均衡数据集, 过采样, 数据清洗, 特征选择, 集成算法, 多分类

CUI Xin, XU Hua, ZHU Liang. Multi-classification Ensemble Algorithm for Imbalanced Data[J]. Computer Engineering and Applications, 2022, 58(2): 176-183.

崔鑫, 徐华, 朱亮. 面向不均衡数据的多分类集成算法[J]. 计算机工程与应用, 2022, 58(2): 176-183.

References

[1] GUO H P，ZHOU J，WU C A.Imbalanced learning based on data-partition and SMOTE[J].Information，2018，9（9）：238.
[2] SUSAN S，KUMAR A.SSOMaj-SMOTE-SSOMin：three-step intelligent pruning of majority and minority samples for learning from imbalanced datasets[J].Applied Soft Computing，2019，78：141-149.
[3] HA J，LEE J S.A new under-sampling method using genetic algorithm for imbalanced data classification[C]//IMCOM’16：Proceedings of the 10th International Conference on Ubiquitous Information Management and Communication.NY，United States：Association for Computing Machinery，2016：1-6.
[4] RAYHAN F，AHMED S，MAHBUB A，et al.CUSBoost：cluster-based under-sampling with boosting for imbalanced classification[C]//2017 2nd International Conference on Computational Systems and Information Technology for Sustainable Solution（CSITSS），Bangalore，India，2017：1-5.
[5] CHAWLA N V，BOWYER K W，HALL L O，et al.SMOTE：synthetic minority over-sampling technique[J].Journal of Artificial Intelligence Research，2011，16（1）：321-357.
[6] 古平，杨炀.面向不均衡数据集中少数类细分的过采样算法[J].计算机工程，2017，43（2）：241-247.
GU P，YANG Y.Oversampling algorithm oriented to subdivision of minority class in imbalanced data set[J].Computer Engineering，2017，43（2）：241-247.
[7] 杨毅，卢诚波，徐根海.面向不平衡数据集的一种精化Borderline-SMOTE方法[J].复旦学报（自然科学版），2017，56（5）：537-544.
YANG Y，LU C B，XU G H.A refined Borderline-SMOTE method for imbalanced data set[J].Journal of Fudan University（Natural Science），2017，56（5）：537-544.
[8] HE H，BAI Y，GARCIA E A，et al.ADASYN：adaptive synthetic sampling approach for imbalanced learning[C]//2008 IEEE International Joint Conference on Neural Networks（IEEE World Congress on Computational Intelligence），Hong Kong，China，2008：1322-1328.
[9] 赵清华，张艺豪，马建芬，等.改进SMOTE的非平衡数据集分类算法研究[J].计算机工程与应用，2018，54（18）：168-173.
ZHAO Q H，ZHANG Y H，MA J F，et al.Research on classification algorithm of imbalanced datasets based on improved SMOTE[J].Computer Engineering and Applications，2018，54（18）：168-173.
[10] 易未，毛力，孙俊，等.改进Smote算法在不平衡数据集上的分类研究[J].计算机与现代化，2018（3）：83-88.
YI W，MAO L，SUN J.Research on classification of improved Smote algorithm on imbalanced datasets[J].Computer and Modernization，2018（3）：83-88.
[11] ZHAO H，LI X J.A cost sensitive decision tree algorithm based on weighted class distribution with batch deleting attribute mechanism[J].Information Sciences，2017，378：303-316.
[12] PEREZ-RODRIGUEZ J，ARROYO-PENA A G，GARCIA-PEDRAJAS N.Simultaneous instance and feature selection and weighting using evolutionary computation：proposal and study[J].Applied Soft Computing，2015，37：416-443.
[13] 王馨月，景丽萍.基于分层抽样的不均衡数据集成分类[J].深圳大学学报（理工版），2019，36（1）：24-32.
WANG X Y，JING L P.Stratified sampling based ensemble classification for imbalanced data[J].Journal of Shenzhen University（Science & Engineering），2019，36（1）：24-32.
[14] GUO H X，LI Y J，LI Y A，et al.BPSO-Adaboost-KNN ensemble learning algorithm for multi-class imbalanced data classification[J].Engineering Applications of Artificial Intelligence，2016，49：176-193.
[15] LIU Y，WANG Y Z，REN X G，et al.A classification method based on feature selection for imbalanced[J].IEEE Access，2019，7：81794-81807.
[16] KUNCHEVA L I，WHITAKER C J.Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy[J].Machine Learning，2003，51（2）：181-207.
[17] 姚旭，王晓丹，张玉玺，等.基于自适应t分布变异的粒子群特征选择方法[J].系统工程与电子技术，2013，35（6）：1335-1341.
YAO X，WANG X D，ZHANG Y X，et al.Feature selection particle swarm optimization t distribution mutual information[J].Systems Engineering and Electronics，2013，35（6）：1335-1341.
[18] HASSAN M R，RAMAMOHANARAO K，KARMAKAR C K，et al.A novel scalable multi-class ROC for effective visualization and computation[C]//PAKDD’10：Proceedings of the 14th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining.Berlin，Heidelberg：Springer-Verlag，2010：107-120.
[19] 武森，刘露，卢丹.基于聚类欠采样的集成不均衡数据分类算法[J].工程科学学报，2017，39（8）：1244-1253.
WU S，LIU L，LU D.Imbalanced data ensemble classification based on cluster-based under-sampling algorithm[J].Journal of University of Science and Technology Beijing，2017，39（8）：1244-1253.
[20] LIN W C，TSAI C，HU Y H，et al.Clustering-based undersampling in class-imbalanced data[J].Information Sciences，2017，409/410：17-26.
[21] KRAWCZYK B，WO?NIAK M，SCHAEFER G.Cost-sensitive decision tree ensembles for effective imbalanced classification[J].Applied Soft Computing，2014，14（1）：554-562.