粒向量驱动的随机森林分类算法研究

doi:10.3778/j.issn.1002-8331.2305-0204

摘要/Abstract

摘要： 粒计算是一种符合人类认知特性的计算范式，能够有效处理复杂数据。随机森林通过集成多个决策树来降低单个决策树的过拟合风险，但仍存在一定的过拟合问题。为了减少过拟合并提高分类性能，在随机森林中引入了粒向量表示，提出了基于粒向量的随机森林分类算法。粒向量具有表示高维特征的能力，可以捕捉更多的数据模式；参照样本选择的随机性有助于控制过拟合；不同决策树使用不同的粒向量可以增加模型的多样性。实验结果表明，与传统随机森林以及其他改进算法相比，基于粒向量表示的随机森林算法具有较好的泛化能力，有效提高了分类的准确率，表明了基于粒向量的随机森林分类算法的正确性与有效性。

关键词: 粒计算, 粒向量, 随机森林, 集成学习

Abstract: Granular computing is a computational paradigm that aligns with human cognitive characteristics, enabling the effective processing of complex data. Random Forest reduces the risk of overfitting associated with individual decision trees by ensembling multiple trees. However, it still faces some overfitting issues. To mitigate overfitting and enhance classification performance, this research introduces the concept of granular vector representation into Random Forest. Granular vectors possess the ability to represent high-dimensional features, thereby capturing more data patterns. The randomness in sample selection aids in controlling overfitting, while using different granular vectors for distinct decision trees enhances model diversity. Experimental results demonstrate that, compared to traditional Random Forest and other enhanced algorithms, the Random Forest algorithm based on granular vector representation exhibits superior generalization capabilities and significantly improves classification accuracy. The correctness and effectiveness of the granular vector-based Random Forest classification algorithm are demonstrated.

Key words: granular computing, granular vector, random forest, ensemble learning

张锟滨, 陈玉明, 吴克寿, 侯贤宇. 粒向量驱动的随机森林分类算法研究[J]. 计算机工程与应用, 2024, 60(3): 148-156.

ZHANG Kunbin, CHEN Yumin, WU Keshou, HOU Xianyu. Research on Granule Vectors Random Forest Classification Algorithm[J]. Computer Engineering and Applications, 2024, 60(3): 148-156.

参考文献

[1] ZADEH L A, GUPTA M M, RAGADE R K, et al. Fuzzy sets and information granularity[M]//Advances in fuzzy set theory and applications. [S.l.]: North Holland Publishing Company, 1979: 3-18.
[2] LIN T Y. Neighborhood systems and relational databases[C]//Proceedings of the 1988 ACM Sixteenth Annual Conference on Computer Science, 1988: 725.
[3] LIN T Y, ZADEH L A. Special issue on granular computing and data mining[J]. International Journal of Intelligent Systems, 2004, 19(7): 565-566.
[4] YAO Y Y. Information granulation and rough set approximation[J]. International Journal of Intelligent Systems, 2001, 16(1): 87-104.
[5] YAO Y Y. Relational interpretations of neighborhood operators and rough set approximation operators[J]. Information Sciences, 1998, 111(1/4): 239-259.
[6] 苗夺谦, 范世栋. 知识的粒度计算及其应用[J]. 系统工程理论与实践, 2002, 22(1): 48-56.
MIAO D Q, FAN S D. The calculation of knowledge granulation and its application[J]. Systems Engineering-Theory & Practice, 2002, 22(1): 48-56.
[7] 胡清华, 于达仁, 谢宗霞. 基于邻域粒化和粗糙逼近的数值属性约简[J]. 软件学报, 2008, 19(3): 640-649.
HU Q H, YU D R, XIE Z X. Numerical attribute reduction based on neighborhood granulation and rough approximation[J]. Journal of Software, 2008, 19(3): 640-649.
[8] HU Q H, YU D R, XIE Z X Neighborhood classifiers[J]. Expert Systems with Applications, 2008, 34(2): 866-876.
[9] ZHU P F, HU Q H, HAN Y H, et al. Combining neighborhood separable subspaces for classification via sparsity regularized optimization[J]. Information Sciences, 2016, 370-371(1): 270-287.
[10] CHEN Y, ZHU S, LI W, et al. Fuzzy granular convolutional classifiers[J]. Fuzzy Sets and Systems, 2022, 426: 145-162.
[11] LI W, MA X Y, CHEN Y M, et al. Random fuzzy granular decision tree[J]. Mathematical Problems in Engineering, 2021(10): 1-17.
[12] CHEN Y, QIN N, LI W, et al. Granule structures, distances and measures in neighborhood systems[J]. Knowledge-Based Systems, 2019, 165: 268-281.
[13] 卜东波, 白硕, 李国杰. 聚类/分类中的粒度原理[J]. 计算机学报, 2002, 25(8): 810-816.
BU D B, BAI S, LI G J. Principle of granularity in clustering/classification[J]. Chinese Journal of Computers, 2002, 25(8): 810-816.
[14] 王国胤, 张清华, 胡军. 粒计算研究综述[J]. 智能系统学报, 2007, 2(6): 8-26.
WANG G Y, ZHANG Q H, HU J. An overview of granular computing[J]. CAAI Transactions on Intelligent Systems, 2007, 2(6): 8-26.
[15] DAI J, HAN H, ZHANG X, et al. Catoptrical rough set model on two universes using granule-based definition and its variable precision extensions[J]. Information Sciences, 2017(390): 70-81.
[16] ROH S B, OH S K, PEDRYCZ W. A fuzzy ensemble of parallel polynomial neural networks with information granules formed by fuzzy clustering[J]. Knowledge-Based Systems, 2010, 23(3): 202-219.
[17] WU W Z, LEUNG Y. Theory and applications of granular labelled partitions in multi-scale decision tables[J]. Information Sciences, 2011, 181(18): 3878-3897.
[18] BREIMAN L. Random forest[J]. Machine Learning, 2001, 45: 5-32.
[19] 任家东, 刘新倩, 王倩, 等. 基于KNN离群点检测和随机森林的多层入侵检测方法[J]. 计算机研究与发展, 2019, 56(3): 566-575.
   REN J D, LIU X Q, WANG Q, et alL. A multi-level intrusion detection method based on KNN outlier detection and random forests[J]. Journal of Computer Research and Development, 2019, 56(3): 566-575.
[20] 沈晶磊, 虞慧群, 范贵生, 等. 基于随机森林算法的推荐系统的设计与实现[J]. 计算机科学, 2017, 44(11): 164-167.
   SHEN J L, YU H Q, FAN G S, et al. Design and implementation of a recommendation system based on random forest algorithm[J]. Computer Science, 2017, 44(11): 164-167.
[21] 李旭明, 李传军. 基于随机森林模型的输电线路故障检测系统研究[J]. 计算技术与自动化, 2020, 39(1): 29-33.
   LI X M, LI C J. Research on transmission line fault detection system based on random forest model[J]. Computing Technology and Automation, 2020, 39(1): 29-33.
[22] 方匡南, 吴见彬, 朱建平, 等. 随机森林方法研究综述[J]. 统计与信息论坛, 2011, 26(3): 32-38.
   FANG K N, WU J B, ZHU J P, et al. A review of random forest method[J]. Statistics and Information Forum, 2011, 26(3): 32-38.
[23] 徐计, 王国胤, 于洪. 基于粒计算的大数据处理[J]. 计算机学报, 2015, 38(8): 1497-1517.
   XU J, WANG G Y, YU H, Big data processing based on granular computing[J]. Chinese Journal of Computers, 2015, 38(8): 1497-1517.
[24] GEURTS P, ERNST D, WEHENKEL L. Extremely randomized trees[J]. Machine Learning, 2006, 63(1): 3-42.
[25] RODRíGUEZ D, JUAN J, LUDMILA I. K. , et al, Rotation forest: a new classifier ensemble method[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28: 1619-1630.
[26] CHEN T, GUESTRIN C. Xgboost: a scalable tree boosting system[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016: 785-794.