Similarity-Based Training Set Recommendation for Software Defect Prediction

doi:10.3778/j.issn.1002-8331.2208-0297

Abstract

Abstract: In the process of software defect prediction, the quality of training set is the key factor affecting the prediction results. In recent years, training set selection has also become a research hotspot in cross-project defect prediction and cross-version defect prediction scenarios. However, most of the existing studies focus on a single prediction scenario, which may affect the quality of training set to some extent. Based on two scenarios of cross-project defect prediction and cross-version defect prediction, this paper proposes an approach of similarity-based training set recommendation（STSR） from the perspective of data distribution. Firstly, the candidate source projects and target project are divided into the same number of clusters by clustering, and the Euclidean distances between the cluster centers are calculated to measure the similarity of datasets. Then, the target project is sampled, and the defect rate differences between the candidate source projects and sampled target project are calculated. Moreover, the interference class ratio is calculated. Finally, the recommendation of training set is realized. The experiments are conducted on 40 versions of 11 projects from the PROMISE repository, and the performance of STSR is evaluated by F1 and AUC. The results indicate that the performance of STSR is better than that of cross-version defect prediction under F1, and similar under AUC. In terms of time cost, the maximum recommended time of STSR method is 5.09?s, which is acceptable.

Key words: software defect prediction, training set recommendation, cross-project defect prediction, cross-version defect prediction

摘要： 在软件缺陷预测过程中，训练集质量是影响预测结果的关键因素。近几年，训练集选择也成为跨项目缺陷预测和跨版本缺陷预测等场景下的研究热点。然而，现有研究大多针对单一预测场景，可能会在一定程度上影响训练集质量。基于跨项目缺陷预测和跨版本缺陷预测两个场景，从数据分布角度提出一种基于相似性度量的训练集推荐（similarity-based training set recommendation，STSR）方法。采用聚类将候选源项目与目标项目划分为相同个数的簇，计算簇心之间的欧氏距离衡量数据集的相似度，对目标项目进行抽样，计算候选源项目与抽样目标项目缺陷率的差值，并计算干扰类比率，最终实现训练集推荐。在PROMISE数据集的11个项目共40个版本上进行实验验证，采用F1和AUC指标评价STSR方法的性能。实验结果表明，与跨版本缺陷预测相比，STSR方法的F1更优，在AUC指标上两者相当；在时间代价方面，STSR方法的最长推荐时间为5.09?s，是可接受的。

关键词: 软件缺陷预测, 训练集选择, 跨项目缺陷预测, 跨版本缺陷预测

WANG Chao, YU Qiao, HAN Hui. Similarity-Based Training Set Recommendation for Software Defect Prediction[J]. Computer Engineering and Applications, 2023, 59(9): 86-94.

王朝, 于巧, 韩惠. 基于相似性度量的软件缺陷预测训练集推荐[J]. 计算机工程与应用, 2023, 59(9): 86-94.

References

[1] 陈翔，王莉萍，顾庆，等.跨项目软件缺陷预测方法研究综述[J].计算机学报，2018，41（1）：254-274.
CHEN X，WANG L P，GU Q，et al.A survey on cross-project software defect prediction methods[J].Chinese Journal of Computers，2018，41（1）：254-274.
[2] 吴方君.静态软件缺陷预测研究进展[J].计算机科学与探索，2019，13（10）：1621-1637.
WU F J.Research progress of static software defect prediction[J].Journal of Frontiers of Computer Science and Technology，2019，13（10）：1621-1637.
[3] 宫丽娜，姜淑娟，姜丽.软件缺陷预测技术研究进展[J].软件学报，2019，30（10）：3090-3114.
GONG L N，JIANG S J，JIANG L.Research progress of software defect prediction[J].Journal of Software，2019，30（10）：3090-3114.
[4] ZIMMERMANN T，NAGAPPAN N，GALL H，et al.Cross-project defect prediction：a large scale experiment on data vs.domain vs.process[C]//Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering，Amsterdam，Aug 24-28，2009.New York：ACM，2009：91-100.
[5] NAM J，PAN S J，KIM S.Transfer defect learning[C]//Proceedings of the 35th International Conference on Software Engineering，San Francisco，May 18-26，2013.Washington：IEEE Computer Society，2013：382-391.
[6] LIU C，YANG D，XIA X，et al.A two-phase transfer learning model for cross-project defect prediction[J].Information and Software Technology，2019，107：125-136.
[7] SUN Z，LI J，SUN H，et al.CFPS：collaborative filtering based source projects selection for cross-project defect prediction[J].Applied Soft Computing，2021，99：106940.
[8] AMASAKI S.Cross-version defect prediction using cross-project defect prediction approaches：does it work?[C]//Proceedings of the 14th International Conference on Predictive Models and Data Analytics in Software Engineering，Finland，Oct 10，2018.New York：ACM，2018：32-41.
[9] YU Q，ZHU Y，HAN H，et al.Evolutionary measures for object-oriented projects and impact on the performance of cross-version defect prediction[C]//Proceedings of the 13th Asia-Pacific Symposium on Internetware，Hohhot，June 11-12，2022.New York：ACM，2022：192-201.
[10] HERBOLD S.Training data selection for cross-project defect prediction[C]//Proceedings of the 9th International Conference on Predictive Models in Software Engineering，Baltimore，Oct 9，2013.New York：ACM，2013：1-10.
[11] 赵宇，祝义，于巧，等.基于分层数据筛选的跨项目缺陷预测方法[J].计算机工程与应用，2020，57（20）：279-286.
ZHAO Y，ZHU Y，YU Q，et al.Cross project defect prediction method based on hierarchical data screening[J].Computer Engineering and Applications，2020，57（20）：279-286.
[12] ZHENG S，GAI J，YU H，et al.Training data selection for imbalanced cross-project defect prediction[J].Computers & Electrical Engineering，2021，94：107370.
[13] LU H H，KOCAGUNELI E，CUKIC B.Defect prediction between software versions with active learning and dimensionality reduction[C]//Proceedings of the 25th International Symposium on Software Reliability Engineering，Naples，Nov 3-6，2014.Washington：IEEE Computer Society，2014：312-322.
[14] XU Z，LIU J，LUO X，et al.Cross-version defect prediction via hybrid active learning with kernel principal component analysis[C]//Proceedings of the 25th International Conference on Software Analysis，Evolution and Reengineering，Campobasso，March 20-23，2018.Washington：IEEE Computer Society，2018：209-220.
[15] XU Z，LI S，LUO X，et al.TSTSS：a two-stage training subset selection framework for cross version defect prediction[J].Journal of Systems and Software，2019，154：59-78.
[16] AMASAKI S.Cross-version defect prediction：use historical data，cross-project data，or both?[J].Empirical Software Engineering，2020，25（2）：1573-1595.
[17] ZHAO Y，WANG Y，ZHANG Y，et al.ST-TLF：cross-version defect prediction framework based transfer learning[J].Information and Software Technology，2022，149：106939.
[18] 倪超，陈翔，刘望舒，等.基于特征迁移和实例迁移的跨项目缺陷预测方法[J].软件学报，2019，30（5）：1308-1329.
NI C，CHEN X，LIU W S，et al.Cross-project defect prediction method based on feature transfer and instance transfer[J].Journal of Software，2019，30（5）：1308-1329.
[19] YU Q，JIANG S J，QIAN J Y，et al.Process metrics for software defect prediction in object-oriented programs[J].IET Software，2020，14（3）：283-292.
[20] LE CESSIE S，VAN HOUWELINGEN J C.Ridge estimators in logistic regression[J].Journal of the Royal Statistical Society，1992，41（1）：191-201.
[21] JOHN G H，LANGLEY P.Estimating continuous distributions in Bayesian classifiers[C]//Proceedings of the 11th Annual Conference on Uncertainty in Artificial Intelligence，Montreal，Aug 18-20，1995.San Francisco：Morgan Kaufmann，1995：338-345.
[22] BREIMAN L.Random forests[J].Machine Learning，2001，45（1）：5-32.
[23] GUYON I，ELISSEEFF A.An introduction to variable and feature selection[J].Journal of Machine Learning Research，2003，3：1157-1182.
[24] KAREGOWDA A G，MANJUNATH A S，JAYARAM M A.Comparative study of attribute selection using gain ratio and correlation based feature selection[J].International Journal of Information Technology and Knowledge Management，2010，2（2）：271-277.
[25] KONONENKO I.Estimating attributes：analysis and extensions of RELIEF[C]//Proceedings of the European Conference on Machine Learning，Catania，April 6-8，1994.Berlin，Heidelberg：Springer，1994：171-182.