Multi-Instance Ensemble Algorithm Combined with Fuzzy Clustering

doi:10.3778/j.issn.1002-8331.2103-0520

Abstract

Abstract: To solve the problem that many algorithms make assumptions about the proportion of positive instances in the positive bags, a multi-instance ensemble algorithm combined with fuzzy clustering is proposed. Firstly, combining the fuzzy clustering and the characteristics of negative bags in multi-instance learning, the concept of positive score is proposed to measure the possibility of instance’s label being positive, which can reduce the ambiguity of instance’s label in multi-instance learning. Then, considering that it is more costly to classify negative instances incorrectly in multi-instance learning, an instance selection strategy of bag representative is designed, and the selected representative instances are used as the training subsets of the base classifiers. Finally, the results of each base classifier are combined to determine the final label of the bag. The ISFC algorithm does not make any assumption about the proportion of positive instances in positive bags, and can solve the class imbalanced problem when the number of positive bags is large and the number of negative bags is small. Experimental results show that ISFC has achieved good classification effect in drug molecular activity prediction, image classification, and text classification tasks.

Key words: multi-instance learning, fuzzy clustering, random subspace, instance selection, ensemble learning

摘要： 针对许多多示例算法都对正包中的示例情况做出假设的问题，提出了结合模糊聚类的多示例集成算法（ISFC）。结合模糊聚类和多示例学习中负包的特点，提出了“正得分”的概念，用于衡量示例标签为正的可能性，降低了多示例学习中示例标签的歧义性；考虑到多示例学习中将负示例分类错误的代价更大，设计了一种包的代表示例选择策略，选出的代表示例作为基分类器的训练子集；结合各基分类器的结果，确定包的最终标签。ISFC算法对正包中正示例的比例未做任何假设，同时能够解决正包数量多、负包数量少情况下的类别不平衡问题。实验结果表明，ISFC在药物分子活性预测、图像分类、文本分类任务上都取得了较好的分类效果。

关键词: 多示例学习, 模糊聚类, 随机子空间, 示例选择, 集成学习

HAN Haiyun, YANG Youlong, SUN Liqin. Multi-Instance Ensemble Algorithm Combined with Fuzzy Clustering[J]. Computer Engineering and Applications, 2022, 58(7): 87-96.

韩海韵, 杨有龙, 孙丽芹. 结合模糊聚类的多示例集成算法[J]. 计算机工程与应用, 2022, 58(7): 87-96.

References

[1] DIETTERICH T G.Solving the multiple instance problem with axis-parallel rectangles[J].Artificial Intelligence，1997，89：31-71
[2] MARON O，LOZANOPéREZ T.A framework for multiple-instance learning[M].[S.l.]：MIT Press，1998：570-576.
[3] ANDREWS S，TSOCHANTARIDIS I，HOFMANN T.Support vector machines for multiple-instance learning[J].Advances in Neural Information Processing Systems，2003，15（2）：561-568.
[4] CARBONNEAU M A，GRANGER E，RAYMONO A J，et al.Robust multiple?instance learning ensembles using random subspace instance selection[J].Pattern Recognition，2016，58：83-99.
[5] WANG J，ZUCKER J D.Solving multiple-instance problem：A lazy learning approach[C]//Proceedings of the 17th International Conference on Machine Learning，2000.
[6] ZHOU Z H，ZHANG M L.Solving multi-instance problems with classifier ensemble based on constructive clusteing[J].Knowledge & Information System，2007，11（2）：155-170.
[7] CHEPLYGINA V，TAX D M.Multiple instance learning with bag dissimilarities[J].Pattern Recognition，2015，48（1）：264-275.
[8] COATES A，NG A Y.Learning feature representations with [k]-means[J].Lecture Notes in Computer Science，2012，7700：561-580.
[9] CHEN Y，BI J，WANG J Z.MILES：Multiple-instance learning via embedded instance selection[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2006，28（12）：1931.
[10] FU Z，ROBLES-KELLY A，ZHOU J.MILIS：Multiple instance learning with instance selection[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2011，33（5）：958-977.
[11] WU J，PAN S，ZHU X，et al.Multi-instance learning with discriminative bag mapping[J].IEEE Transactions on Knowledge and Data Engineering，2018，30（6）：1065-1080.
[12] YUAN L，WEN X，LU Z，et al.An iterative instance selection based framework for multiple-instance learning[C]//Proceedings of the 2018 IEEE 30th International Conference on Tools with Artificial Intelligence（ICTAI），2018.
[13] DONG L.A comparison of multi-instance learning algorithms[D].Hamilton：The University of Waikato，2006.
[14] RAY S，CRAVEN M.Supervised versus multiple instance learning：An empirical comparison[C]//Proceedings of the Twenty-Second International Conference on Machine Learning，Bonn，Germany，August 7-11，2005.
[15] LI F.Convex multiple-instance learning by estimating likelihood ratio[C]//Advances in Neural Information Processing Systems，2010：1360-1368.
[16] YAN L，TAX D，DUIN R，et al.Multiple-instance learning as a classifier combining problem[J].Pattern Recognition，2013，46（3）：865-874.
[17] BREIMAN L.Bagging predictors[J].Machine Learning，1996，24：123-140.
[18] HO T K.The random subspace method for constructing decision forests[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，1998，20（8）：832-844.
[19] BEZDEK J C.Pattern recognition with fuzzy objective function algorithms[M].[S.l.]：Plenum Press，1981.
[20] DUNN J C.A fuzzy relative of the isodata process and its use in detecting compact well-separated clusters[J].Journal of Cybernetics，1973，3（3）：32-57.
[21] WOLD S，ESBENSEN K，GELADI P.Principal component analysis[J].Chemometrics & Intelligent Laboratory Systems，1987，2（1/3）：37-52.
[22] MELKI G，CANO A，VENTURA S，et al.MIRSVM：Multi-instance support vector machine with bag representatives[J].Pattern Recognition，2018，79：228-241.
[23] LU J，MA S.Region-based image annotation using heuristic support vector machine in multiple-instance learning[C]//Proceedings of IEEE Computer Society Conference on Computer Vision & Pattern Recognition，2006.
[24] ZHOU Z H，SUN Y Y，LI Y F.Multi-instance learning by treating instances as non-I.I.D.samples[C]//Proceedings of the 26th Annual International Conference on Machine Leaning，2009：1249-1256.
[25] VIOLA P A，PLATT J C，ZHANG C.Multiple instance boosting for object detection[C]//Advances in Neural Information Processing Systems，2005.