时序因素对即时软件缺陷预测性能影响的实证研究

doi:10.3778/j.issn.1002-8331.2403-0221

摘要/Abstract

摘要： 即时软件缺陷预测是针对开发者提交的代码变更是否存在缺陷进行预测。近年来，由于其细粒度、即时性、易追溯的特点，即时软件缺陷预测成为了缺陷预测领域的研究热点。代码变更提交具有时间特性，然而，现有研究大多忽略了时序因素对即时软件缺陷预测的影响。因此，探究代码变更提交时间对即时软件缺陷预测性能的影响规律具有重要意义。探究了时序因素对项目内和跨项目即时软件缺陷预测性能的影响，采用随机森林、CNN和XGBoost三种模型在9个即时软件缺陷预测数据集上展开了实证研究。研究结果表明：在项目内缺陷预测中，训练集与测试集时间越接近，模型性能越好；与非时序场景相比，时序场景下的跨项目缺陷预测与项目内缺陷预测的性能差距更小。因此，在即时软件缺陷预测研究中应该充分考虑时序因素的影响，在进行训练集的选择时应优先考虑与测试集时间相距较近的数据集。

关键词: 即时软件缺陷预测（JIT-SDP）, 时序因素, 跨项目缺陷预测

Abstract: Just-in-time software defect prediction aims to predict whether code changes submitted by developer contain defects. In recent years, due to its fine-grained, immediacy, and easy traceability, just-in-time software defect prediction has become a hot research topic in the field of defect prediction. Code change commits have time-series characteristic. However, most of the existing researches ignore the impact of time-series factor on the performance of just-in-time software defect prediction. Therefore, exploring the impact of code change submission time on the performance of just-in-time software defect prediction is of great significance. This paper explores the impact of time-series factor on the performance of within-project and cross-project just-in-time software defect prediction. The paper conducts empirical studies on nine datasets with three models, Random Forest, CNN, and XGBoost for just-in-time software defect prediction. The results indicate that, in within-project defect prediction, the closer the time between the training and the testing set, the better the model performance. Compared with non-time-series scenario, the performance gap between cross-project and within-project defect prediction in the time-series scenario is smaller. Therefore, the impact of time-series factors should be fully considered in the research of just-in-time software defect prediction, and priority should be given to datasets closer in time to the testing set when selecting training sets.

Key words: just-in-time software defect prediction (JIT-SDP), time-series, cross-project defect prediction

张雨, 于巧, 祝义, 姜淑娟, 张淑涛. 时序因素对即时软件缺陷预测性能影响的实证研究[J]. 计算机工程与应用, 2025, 61(14): 362-376.

ZHANG Yu, YU Qiao, ZHU Yi, JIANG Shujuan, ZHANG Shutao. Empirical Study on Impact of Time-Series Factor on Performance of Just-in-Time Software Defect Prediction[J]. Computer Engineering and Applications, 2025, 61(14): 362-376.

参考文献

[1] 陈翔, 顾庆, 刘望舒, 等. 静态软件缺陷预测方法研究[J]. 软件学报, 2016, 27(1): 1-25.
CHEN X, GU Q, LIU W S, et al. Survey of static software defect prediction[J]. Journal of Software, 2016, 27(1): 1-25.
[2] MENZIES T, BUTCHER A, COK D, et al. Local versus global lessons for defect prediction and effort estimation[J]. IEEE Transactions on Software Engineering, 2012, 39(6): 822-834.
[3] XIA X, LO D, PAN S J, et al. HYDRA: massively compositional model for cross-project defect prediction[J]. IEEE Transactions on Software Engineering, 2016, 42(10): 977-998.
[4] NAM J, KIM S. Heterogeneous defect prediction[C]//Proceedings of the 10th Joint Meeting on Foundations of Software Engineering. New York: ACM, 2009: 508-519.
[5] ZHANG F, HASSAN A E, MCINTOSH S, et al. The use of summation to aggregate software metrics hinders the performance of defect prediction models[J]. IEEE Transactions on Software Engineering, 2016, 43(5): 476-491.
[6] KIM S, WHITEHEAD E J, ZHANG Y. Classifying software changes: clean or buggy?[J]. IEEE Transactions on Software Engineering, 2008, 34(2): 181-196.
[7] KAMEI Y, SHIHAB E, ADAMS B, et al. A large-scale empirical study of just-in-time quality assurance[J]. IEEE Transactions on Software Engineering, 2012, 39(6): 757-773.
[8] 蔡亮, 范元瑞, 鄢萌, 等. 即时软件缺陷预测研究进展[J]. 软件学报, 2019, 30(5): 1288-1307.
CAI L, FAN Y R, YAN M, et al. Just-in-time software defect prediction: literature review[J]. Journal of Software, 2019, 30(5): 1288-1307.
[9] KAMEI Y, FUKUSHIMA T, MCINTOSH S, et al. Studying just-in-time defect prediction using cross-project models[J]. Empirical Software Engineering, 2016, 21: 2072-2106.
[10] ZHANG W, LI W, JIA X. Effort-aware tri-training for semi-supervised just-in-time defect prediction[C]//Proceedings of the 23rd Pacific-Asia Conference on Knowledge Discovery and Data Mining. Berlin: Springer, 2019: 293-304.
[11] CATOLINO G, DI NUCCI D, FERRUCCI F. Cross-project just-in-time bug prediction for mobile apps: an empirical assessment[C]//Proceedings of the IEEE/ACM 6th International Conference on Mobile Software Engineering and Systems. Piscataway: IEEE, 2019: 99-110.
[12] TRAUTSCH A, HERBOLD S, GRABOWSKI J. Static source code metrics and static analysis warnings for fine-grained just-in-time defect prediction[C]//Proceedings of the IEEE International Conference on Software Maintenance and Evolution. Piscataway: IEEE, 2020: 127-138.
[13] FALESSI D, HUANG J, NARAYANA L, et al. On the need of preserving order of data when validating within-project defect classifiers[J]. Empirical Software Engineering, 2020, 25: 4805-4830.
[14] ZHU K, ZHANG N, YING S, et al. Within‐project and cross‐project just‐in‐time defect prediction based on denoising autoencoder and convolutional neural network[J]. IET Software, 2020, 14(3): 185-195.
[15] PORNPRASIT C, TANTITHAMTHAVORN C, JIARPAKDEE J, et al. PyExplainer: explaining the predictions of just-in-time defect models[C]//Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering. Piscataway: IEEE, 2021: 407-418.
[16] ZHAO K, XU Z, ZHANG T, et al. Simplified deep forest model based just-in-time defect prediction for Android mobile apps[J]. IEEE Transactions on Reliability, 2021, 70(2): 848-859.
[17] YANG X, YU H, FAN G, et al. DEJIT: a differential evolution algorithm for effort-aware just-in-time software defect prediction[J]. International Journal of Software Engineering and Knowledge Engineering, 2021, 31(3): 289-310.
[18] CHEN L, SONG S, WANG C. A novel effort measure method for effort-aware just-in-time software defect prediction[J]. International Journal of Software Engineering and Knowledge Engineering, 2021, 31(8): 1145-1169.
[19] LIN D, TANTITHAMTHAVORN C, HASSAN A E. The impact of data merging on the interpretation of cross-project just-in-time defect models[J]. IEEE Transactions on Software Engineering, 2021, 48(8): 2969-2986.
[20] AMASAKI S, AMAN H, YOKOGAWA T. A preliminary evaluation of CPDP approaches on just-in-time software defect prediction[C]//Proceedings of the 47th Euromicro Conference on Software Engineering and Advanced Applications. Piscataway: IEEE, 2021: 279-286.
[21] PORNPRASIT C, TANTITHAMTHAVORN C K. JITLine: a simpler, better, faster, finer-grained just-in-time defect prediction[C]//Proceedings of the IEEE/ACM 18th International Conference on Mining Software Repositorie. Piscataway: IEEE, 2021: 369-379.
[22] AMASAKI S, AMAN H, YOKOGAWA T. An evaluation of effort-aware fine-grained just-in-time defect prediction methods[C]//Proceedings of the 48th Euromicro Conference on Software Engineering and Advanced Applications. Piscataway: IEEE, 2022: 209-216.
[23] NADIM M, MONDAL D, ROY C K. Leveraging structural properties of source code graphs for just-in-time bug prediction[J]. Automated Software Engineering, 2022, 29(1): 27.
[24] ZHANG T, YU Y, MAO X, et al. FENSE: a feature-based ensemble modeling approach to cross-project just-in-time defect prediction[J]. Empirical Software Engineering, 2022, 27(7): 162.
[25] CHENG T, ZHAO K, SUN S, et al. Effort-aware cross-project just-in-time defect prediction framework for mobile apps[J]. Frontiers of Computer Science, 2022, 16(6): 166207.
[26] 陈丽琼, 王璨, 宋士龙. 一种即时软件缺陷预测模型及其可解释性研究[J]. 小型微型计算机系统, 2022, 43(4): 865-871.
CHEN L Q, WANG C, SONG S L. Just-in-time defect prediction model and its Interpretability research[J]. Journal of Chinese Computer System, 2022, 43(4): 865-871.
[27] 葛建, 虞慧群, 范贵生, 等. 面向智能计算框架的即时软件缺陷预测[J]. 软件学报, 2023, 34(9): 3966-3980.
GE J, YU H Q, FAN G S, et al. Just-in-time defect prediction for intelligent computing frameworks[J]. Journal of Software, 2023, 34(9): 3966-3980.
[28] SONG L, MINKU L L. A procedure to continuously evaluate predictive performance of just-in-time software defect prediction models during software development[J]. IEEE Transactions on Software Engineering, 2023, 49(2): 646-666.
[29] CABRAL G G, MINKU L L. Towards reliable online just-in-time software defect prediction[J]. IEEE Transactions on Software Engineering, 2023, 49(3): 1342-1358.
[30] TABASSUM S, MINKU L L, FENG D. Cross-project online just-in-time software defect prediction[J]. IEEE Transactions on Software Engineering, 2023, 49(1): 268-287.
[31] YANG X, YU H, FAN G, et al. An empirical study on progressive sampling for just-in-time software defect prediction[C]//Proceedings of the 7th International Workshop on Quantitative Approaches to Software Quality, 2019: 12-18.
[32] XU H, DUAN R, YANG S, et al. An empirical study on data sampling for just-in-time defect prediction[C]//Proceedings of the International Conference on Artificial Intelligence and Security. Berlin: Springer, 2021: 54-69.
[33] LI Z, DU Q, ZHANG H. et al. An empirical study of data sampling techniques for just-in-time software defect prediction[J]. Automated Software Engineering, 2024, 31(2): 40.
[34] YANG X, LO D, XIA X, et al. TLEL: a two-layer ensemble learning approach for just-in-time defect prediction[J]. Information and Software Technology, 2017, 87: 206-220.
[35] CHEN X, ZHAO Y, WANG Q, et al. MULTI: multi-objective effort-aware just-in-time software defect prediction[J]. Information and Software Technology, 2018, 93: 1-13.
[36] ZIMMERMANN T, NAGAPPAN N, GALL H, et al. Cross-project defect prediction: a large scale experiment on data vs. domain vs. process[C]//Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, 2009: 91-100.
[37] 陈翔, 王莉萍, 顾庆, 等. 跨项目软件缺陷预测方法研究综述[J]. 计算机学报, 2018, 41(1): 254-274.
CHEN X, WANG L P, GU Q. A survey on cross-project software defect prediction methods[J]. Chinese Journal of Computers, 2018, 41(1): 254-274.
[38] HARMAN M, ISLAM S, JIA Y, et al. Less is more: temporal fault predictive performance over multiple hadoop releases[C]//Proceedings of the 6th International Symposium on Search-Based Software Engineering. Berlin: Springer, 2014: 240-246.
[39] TAN M, TAN L, DARA S, et al. Online defect prediction for imbalanced data[C]//Proceedings of the IEEE/ACM 37th IEEE International Conference on Software Engineering. Piscataway: IEEE, 2015: 99-108.
[40] DITZLER G, ROVERI M, ALIPPI C, et al. Learning in nonstationary environments: a survey[J]. IEEE Computational Intelligence Magazine, 2015, 10(4): 12-25.
[41] MCINTOSH S, KAMEI Y. Are fix-inducing changes a moving target? a longitudinal case study of just-in-time defect prediction[C]//Proceedings of the 40th International Conference on Software Engineering. New York: ACM, 2018: 560.
[42] WANG S, MINKU L L, YAO X. A systematic study of online class imbalance learning with concept drift[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(10): 4802-4821.
[43] TABASSUM S, MINKU L L, FENG D, et al. An investigation of cross-project learning in online just-in-time software defect prediction[C]//Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. New York: ACM, 2020: 554-565.
[44] YANG Y, ZHOU Y, LIU J, et al. Effort-aware just-in-time defect prediction: simple unsupervised models could be better than supervised models[C]//Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. New York: ACM, 2016: 157-168.
[45] LIU J, ZHOU Y, YANG Y, et al. Code churn: a neglected metric in effort-aware just-in-time defect prediction[C]//Proceedings of the ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. Piscataway: IEEE, 2017: 11-19.
[46] ZENG Z, ZHANG Y, ZHANG H, et al. Deep just-in-time defect prediction: how far are we?[C]//Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis. New York: ACM, 2021: 427-438.
[47] BREIMAN L. Random forests[J]. Machine Learning, 2001, 45: 5-32.
[48] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
[49] CHEN T, GUESTRIN C. XGBoost: a scalable tree boosting system[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 785-794.