Abstract: Ensemble learning, which integrates multiple weak learners and produces a stronger learner, is one of the key research areas in machine learning. Although a number of algorithms have been proposed for the generation of base learners, these algorithms are usually with low robustness. This study proposes a novel ensemble learning algorithm, namely an Ensemble Learning Algorithm by Consecutively Removing Training Samples（ELACRTS）, which possesses the merits of both boosting and bagging methods. By removing the samples with high confidence from the training set, the training space is gradually reduced, which allows a sufficient learning on the underrepresented samples. The ELACRTS method generates a series of decreasing training subspaces and therefore produces a number of diverse base classifiers. Similar to boosting and bagging, voting is employed for integration of predictions by multiple base classifiers. It employs 10-folds cross validation to assess the performance of the proposed ELACRTS method. Extensive experiments on 8 datasets and 7 base classifiers demonstrate that the ELACRTS algorithm outperforms the boosting and bagging algorithms.

Key words: ensemble learning, base classifier, training subspace, decreasing, confidence level

摘要： 从多个弱分类器重构出强分类器的集成学习方法是机器学习领域的重要研究方向之一。尽管已有多种多样性基本分类器的生成方法被提出，但这些方法的鲁棒性仍有待提高。递减样本集成学习算法综合了目前最为流行的boosting与bagging算法的学习思想，通过不断移除训练集中置信度较高的样本，使训练集空间依次递减，使得某些被低估的样本在后续的分类器中得到充分训练。该策略形成一系列递减的训练子集，因而也生成一系列多样性的基本分类器。类似于boosting与bagging算法，递减样本集成学习方法采用投票策略对基本分类器进行整合。通过严格的十折叠交叉检验，在8个UCI数据集与7种基本分类器上的测试表明，递减样本集成学习算法总体上要优于boosting与bagging算法。

关键词: 集成学习, 基本分类器, 训练子空间, 递减, 置信度