Improved Algorithm of Fuzzy Decision Tree Based on Neural Network

doi:10.3778/j.issn.1002-8331.2010-0045

Abstract

Abstract:

Traditional decision tree finds the decision boundary by recursively dividing the feature space. It gives a “hard” division of the feature space and accurately describes of data. However, when dealing with big data and complex model problems, this precise decision boundary reduces generalization ability of decision tree. In order decision tree automatically acquire inaccurate knowledge, fuzzy theory is introduced into decision tree and in the process of building, neural network is introduced as decision leaf node. An improved algorithm of fuzzy decision tree based on neural network is proposed. In neural network fuzzy decision tree, classifier learning consists two stage：the first stage uses a heuristic algorithm with reduced uncertainty to divide the big data, and stops the growth of fuzzy decision tree until the node dividing ability is below reality threshold. The second stage uses neural network to classify the leaf node with generalization ability. Experiments show that compared with traditional classification learning algorithm, proposed algorithm has higher accuracy, and can determine the size of decision tree through structural adaptation for classification problem of big data and complex patterns.

Key words: decision tree, neural network, fuzzy theory, fuzzy decision tree based on neural network

摘要：

传统决策树通过对特征空间的递归划分寻找决策边界，给出特征空间的“硬”划分。但对于处理大数据和复杂模式问题时，这种精确决策边界降低了决策树的泛化能力。为了让决策树算法获得对不精确知识的自动获取，把模糊理论引进了决策树，并在建树过程中，引入神经网络作为决策树叶节点，提出了一种基于神经网络的模糊决策树改进算法。在神经网络模糊决策树中，分类器学习包含两个阶段：第一阶段采用不确定性降低的启发式算法对大数据进行划分，直到节点划分能力低于真实度阈值[ε]停止模糊决策树的增长；第二阶段对该模糊决策树叶节点利用神经网络做具有泛化能力的分类。实验结果表明，相较于传统的分类学习算法，该算法准确率高，对识别大数据和复杂模式的分类问题能够通过结构自适应确定决策树规模。

关键词: 决策树, 神经网络, 模糊理论, 神经网络模糊决策树

ZHANG Min, PENG Hongwei, YAN Xiaoling. Improved Algorithm of Fuzzy Decision Tree Based on Neural Network[J]. Computer Engineering and Applications, 2021, 57(21): 174-179.

张敏，彭红伟，颜晓玲. 基于神经网络的模糊决策树改进算法[J]. 计算机工程与应用, 2021, 57(21): 174-179.

References

[1] ANITHA M，BHARATHI S D，ILAKIYA K，et al.On distributed fuzzy decision trees for big data[J].IEEE Transactions on Fuzzy Systems，2018，52（99）：11-16.
[2] 张昕，郭阳.复杂网络中的一种介度熵抗毁性度量方法[J].计算机工程与应用，2020，56（12）：105-111.
ZHANG X，GUO Y.Betweenness degree entropy invulnerability measurement method in complex networks[J].Computer Engineering and Applications，2020，56（12）：105-111.
[3] 禹建丽，谷丰盈.基于模糊决策的多响应参数优化方法[J].模糊系统与数学，2019，98（6）：43-55.
YU J L，GU F Y.Multi-response parameter optimization method based on fuzzy decision[J].Fuzzy Systems and Mathematics，2019，98（6）：43-55.
[4] 李海涛，刘泰麟，邵泽东，等.基于决策树与二分分割算法的BP神经网络在赤潮等级预测中的应用研究[J].海洋科学，2019，43（9）：34-40.
LI H T，LIU T L，SHAO Z D，et al.Application research of BP neural network based on decision tree and bisection algorithm in red tide level prediction[J].Marine Science，2019，43（9）：34-40.
[5] WANG R，HE Y L，CHOW C Y，et al.Learning ELM-tree from big data based on uncertainty reduction[J].Fuzzy Sets and Systems，2015，258（33）：270-274.
[6] SEGATORI A，MARCELLONI F，PEDRYCZ W.On distributed fuzzy decision trees for big data[J].IEEE Transactions on Fuzzy Systems，2017，34（6）：101-109.
[7] 安葳鹏，尚家泽.决策树C4.5算法的改进与分析[J].计算机工程与应用，2019，55（12）：169-173.
AN W P，SHANG J Z.Improvement and analysis of C4.5 decision tree algorithm[J].Computer Engineering and Applications，2019，55（12）：169-173.
[8] DíEZ F J，LUQUE M.Decision analysis networks[J].International Journal of Approximate Reasoning，2018，96（2）：83-90.
[9] 姚晟，徐风，赵鹏.基于自适应邻域空间粗糙集模型的直觉模糊熵特征选择[J].计算机研究与发展，2018，55（4）：802-814.
YAO S，XU F，ZHAO P.Intuitionistic fuzzy entropy feature selection based on adaptive neighborhood space rough set model[J].Journal of Computer Research and Development，2018，55（4）：802-814.
[10] LI G H，HAO W J，LI Z W.Generalized consistent space based on distance and its attribute reduction[J].Fuzzy Systems and Mathematics，2018，32（6）：150-154.
[11] ZHOU X，YAN D.Model tree pruning[J].International Journal of Machine Learning and Cybernetics，2019，56（4）：825-835.
[12] FASIHI H，SHAHRAKI M H N.Incremental mining maximal frequent patterns from univariate uncertain data[J].Knowledge-Based Systems，2018，22（3）：326-337.
[13] WU D，LIN C T，HUANG J，et al.On the functional equivalence of TSK fuzzy systems to neural networks，mixture of experts，CART，and stacking ensemble regression[J].IEEE Transactions on Fuzzy Systems，2019，55（11）：55-65.
[14] YIN S，LI P，XU Y，et al.Logistic model-based genetic analysis for kernel filling in a maize RIL population[J].Euphytica，2018，214（5）：86-95.
[15] 张巧荣.利用背景先验的显著性检测算法[J].中国图象图形学报，2016，21（2）：165-173.
ZHANG Q R.Saliency detection algorithm using background prior[J].Journal of Image and Graphics，2016，21（2）：165-173.