基于特征融合的部分有序深度森林模型

doi:10.3778/j.issn.1002-8331.2405-0310

摘要/Abstract

摘要： 部分有序数据是同时包含有序特征与无序特征的一类数据，其广泛存在于现实生活中。传统的有序分类方法或者将所有特征都视为有序特征，或者对有序与无序特征分别进行处理，忽略了二者之间的关系，这些方法难以有效解决部分有序数据上的分类问题。针对该问题，提出一种基于特征融合的部分有序深度森林模型，称为FFDF（feature fusion-based deep forest）。利用典型相关分析的思想，设计特征融合的贡献度计算方法，将有序特征和无序特征融合到同一特征空间，统一度量二者之间的关系。对融合的特征空间进行数据粒化，降低模型处理连续变量时的复杂性。设计融合空间下的特征矩阵输入级联森林，构建部分有序的深度森林模型。在来自UCI和WEKA的13个公共数据集上与部分单调决策树、有序分类模型、深度森林模型等六种方法进行比较实验，结果表明所提方法在准确性和平均绝对误差方面均优于对比方法；与集成模型深度森林gcForest和DF21进行了时间性能上的对比实验，结果表明所提方法在时间性能上优于对比方法。

关键词: 有序分类, 部分有序数据, 特征融合, 深度森林, 典型相关分析

Abstract: Partially ordered data is a class of data that contains both ordered and unordered features, and it exists widely in real life. Traditional ordered classification methods either regard all features as ordered features, or treat ordered and unordered features separately, ignoring the relationship between the two, these methods are difficult to effectively solve the classification problem on partially ordered data. To address the problem, a partially ordered deep forest model based on feature fusion, called FFDF (feature fusion-based deep forest), is proposed. Firstly, the idea of typical correlation analysis is utilized to design the contribution calculation method of feature fusion, which fuses ordered and unordered features into the same feature space, which helps to unify the measure of the relationship between the two. Then, the fused feature space is granulated, which can reduce the complexity of the model when dealing with continuous variable. Finally, the feature matrix on the fusion space is designed to input the cascade forest to construct a partially ordered deep forest model. Comparison experiments with six methods including partial monotone decision tree, ordered classification models REMT, OSDL, OLM, deep forest model gcForest and DF21 on 13 public datasets from UCI and WEKA, show that the proposed method outperforms the comparison methods in terms of accuracy and mean absolute error. In addition, the comparison experiments on time performance are conducted with the ensemble models gcForest and DF21, and the results show that the proposed method outperforms the comparison methods in terms of running efficiency.

Key words: ordered classification, partially ordered data, feature fusion, deep forests, typical correlation analysis

许行, 温萧轲, 王文剑. 基于特征融合的部分有序深度森林模型[J]. 计算机工程与应用, 2025, 61(7): 165-175.

XU Hang, WEN Xiaoke, WANG Wenjian. Partial Ordered Deep Forest Model Based on Feature Fusion[J]. Computer Engineering and Applications, 2025, 61(7): 165-175.

参考文献

[1] DENUIT M M. Risk apportionment and multiply monotone targets[J]. Mathematical Social Sciences, 2018, 92: 74-77.
[2] GONZáLEZ S, GARCíA S, LI S T, et al. Chain based sampling for monotonic imbalanced classification[J]. Information Sciences, 2019, 474: 187-204.
[3] LI Z, LIU G N, LI Q. Nonparametric KNN estimation with monotone constraints[J]. Econometric Reviews, 2017, 36(6/7/8/9): 988-1006.
[4] SANG B B, CHEN H M, LI T R, et al. Incremental approaches for heterogeneous feature selection in dynamic ordered data[J]. Information Sciences, 2020, 541: 475-501.
[5] üNAL F, B?RANT D, ?EKER ?. A new approach: semisupervised ordinal classification[J]. Turkish Journal of Electrical Engineering and Computer Sciences, 2021, 29(3): 1797-1820.
[6] GONZáLEZ S, HERRERA F, GARCíA S. Monotonic random forest with an ensemble pruning mechanism based on the degree of monotonicity[J]. New Generation Computing, 2015, 33(4): 367-388.
[7] 袁方, 杨有龙. 针对混合型分类数据改进的K-modes算法距离公式[J]. 计算机工程与应用, 2020, 56(6): 186-193.
YUAN F, YANG Y L. Improved distance formula of K-modes clustering algorithm for mixed categorical attribute data[J]. Computer Engineering and Applications, 2020, 56(6): 186-193.
[8] WANG X, ZHAI J H, CHEN J K, et al. Ordinal decision trees based on fuzzy rank entropy[C]//Proceedings of the 2015 International Conference on Wavelet Analysis and Pattern Recognition. Piscataway: IEEE, 2015: 208-213.
[9] POTHARST R, FEELDERS A J. Classification trees for problems with monotonicity constraints[J]. ACM SIGKDD Explorations Newsletter, 2002, 4(1): 1-10.
[10] SENGE R, HüLLERMEIER E. Top-down induction of fuzzy pattern trees[J]. IEEE Transactions on Fuzzy Systems, 2010, 19(2): 241-252.
[11] WU G D, ZHU Z W, HUANG P H. A TS-type maximizing-discriminability-based recurrent fuzzy network for classification problems[J]. IEEE Transactions on Fuzzy Systems, 2010, 19(2): 339-352.
[12] GRECO S, MATARAZZO B, SLOWINSKI R. Rough sets methodology for sorting problems in presence of multiple attributes and criteria[J]. European Journal of Operational Research, 2002, 138(2): 247-259.
[13] CANO J R, GUTIéRREZ P A, KRAWCZYK B, et al. Monotonic classification: an overview on algorithms, performance measures and data sets[J]. Neurocomputing, 2019, 341: 168-182.
[14] CHEN C C, LI S T. Credit rating with a monotonicity-constrained support vector machine model[J]. Expert Systems with Applications, 2014, 41(16): 7235-7247.
[15] DOUMPOS M, ZOPOUNIDIS C. A multicriteria decision support system for bank rating[J]. Decision Support Systems, 2010, 50(1): 55-63.
[16] GENEST D, CHEIN M. A content-search information retrieval process based on conceptual graphs[J]. Knowledge and Information Systems, 2005, 8(3): 292-309.
[17] PILTAN M, SOWLATI T. A multi-criteria decision support model for evaluating the performance of partnerships[J]. Expert Systems with Applications, 2016, 45: 373-384.
[18] SCHALL D. A multi-criteria ranking framework for partner selection in scientific collaboration environments[J]. Decision Support Systems, 2014, 59: 1-14.
[19] SOUSA R, YEVSEYEVA I, COSTA J, et al. Multicriteria models for learning ordinal data: a literature review[M]//Artificial intelligence, evolutionary computing and metaheuristics. Berlin, Heidelberg: Springer, 2012: 109-138.
[20] VELIKOVA M, DANIELS H. Decision trees for monotone price models[J]. Computational Management Science, 2004, 1(3): 231-244.
[21] UNAL F, BIRANT D. Educational data mining using semi-supervised ordinal classification[C]//Proceedings of the 2021 3rd International Congress on Human-Computer Interaction, Optimization and Robotic Applications, 2021: 1-5.
[22] 王鑫, 王熙照, 陈建凯, 等. 有序决策树的比较研究[J]. 计算机科学与探索, 2013, 7(11): 1018-1025.
WANG X, WANG X Z, CHEN J K, et al. Comparative study on ordinal decision trees[J]. Journal of Frontiers of Computer Science and Technology, 2013, 7(11): 1018-1025.
[23] HU Q H, CHE X J, ZHANG L, et al. Rank entropy-based decision trees for monotonic classification[J]. IEEE Transactions on Knowledge and Data Engineering, 2012, 24(11): 2052-2064.
[24] LIEVENS S, DE?BAETS B, CAO-VAN K. A probabilistic framework for the design of instance-based supervised ranking algorithms inanordinal setting[J]. Annals of Operations Research, 2008, 163(1): 115-142.
[25] BEN-DAVID A. Automatic generation of symbolic multiattribute ordinal knowledge-based DSSs: methodology and applications[J]. Decision Sciences, 1992, 23(6): 1357-1372.
[26] PEI S L, HU Q H. Partially monotonic decision trees[J]. Information Sciences, 2018, 424: 104-117.
[27] XU H, MA S, WANG W J. An ordered feature recognition method based on ranking separability[J]. Information Sciences, 2023, 648: 119518.
[28] ZHOU Z H, FENG J. Deep forest[J]. National Science Review, 2019, 6(1): 74-86.
[29] GUO Y, LIU S H, LI Z H, et al. BCDForest: a boosting cascade deep forest model towards the classification of cancer subtypes based on gene expression data[J]. BMC Bioinformatics, 2018, 19(5): 1-13.
[30] PANG M, TING K M, ZHAO P, et al. Improving deep forest by screening[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 34(9): 4298-4312.
[31] SUN Q S, ZENG S G, LIU Y, et al. A new method of feature fusion and its application in image recognition[J]. Pattern Recognition, 2005, 38(12): 2437-2448.