research-article

Iterative identification of fault-prone binaries using in-process metrics

Authors:

Gunnar Kudrjavets,

Nachiappan NagappanAuthors Info & Claims

ESEM '08: Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement

Pages 206 - 212

https://doi.org/10.1145/1414004.1414038

Published: 09 October 2008 Publication History

Abstract

Code churn, the amount of code change taking place within a software unit over time, has been correlated with fault-proneness in software systems. We investigate the use of code churn and static metrics collected at regular time intervals during the development cycle to predict faults in an iterative, in-process manner. We collected 159 churn and structure metrics from six, four-month snapshots of a 1 million LOC Microsoft product. The number of software faults fixed during each period is recorded per binary module. Using stepwise logistic regression, we create a prediction model to identify fault-prone binaries using three parameters: code churn (the number of new and changed blocks); class Fan In and class Fan Out (normalized by lines of code). The iteratively-built model is 80.0% accurate at predicting fault-prone and non-fault-prone binaries. These fault-prediction models have the advantage of allowing the engineers to observe how their fault-prediction profile evolves over time.

References

[1]

Alfred V. Aho, Ravi Sethi, and J. D. Ullman, Compilers Principles, Techniques, and Tools. Reading, MA: Addison-Wesley, 1986.

Digital Library

[2]

V. Basili, Briand, L., Melo, W., "A Validation of Object Oriented Design Metrics as Quality Indicators," IEEE Transactions on Software Engineering, vol. Vol. 22, pp. 751--761, 1996.

Digital Library

[3]

L. C. Briand, Wuest, J., Daly, J. W., Porter, D. V., "Exploring the Relationship between Design Measures and Software Quality in Object Oriented Systems," Journal of Systems and Software, vol. Vol. 51, pp. 245--273, 2000.

Digital Library

[4]

L. C. Briand, Wuest, J., Ikonomovski, S., Lounis, H., "Investigating quality factors in object-oriented designs: an industrial case study," in ICSE, 1999, pp. 345--354.

Digital Library

[5]

S. R. Chidamber, Kemerer, C. F., "A Metrics Suite for Object Oriented Design," IEEE Transactions on Software Engineering, vol. 20, pp. 476--493, 1994.

Digital Library

[6]

K. El Emam, S. Benlarbi, N. Goel, and S. N. Rai, "The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics," IEEE Transactions on Software Engineering, vol. 27, pp. 630--650, July 2001.

Digital Library

[7]

T. L. Graves, Karr, A. F., Marron, J. S., Siy, H., "Predicting Fault Incidence Using Software Change History," IEEE Transactions on Software Engineering, vol. 26, pp. 653--661, 2000.

Digital Library

[8]

J. E. Jackson, A User's Guide to Principal Components. New York: Wiley, 1991.

[9]

A. Mockus, Zhang, P., Li, P., "Drivers for customer perceived software quality," in International Conference on Software Engineering (ICSE 05), St. Louis, MO, 2005, pp. 225--233.

Digital Library

[10]

N. Nagappan, Ball, T., "Use of Relative Code Churn Measures to Predict System Defect Density," in International Conference on Software Engineering (ICSE), St. Louis, MO, 2005, pp. 284--292.

Digital Library

[11]

N. Nagappan, Ball, T., Murphy, B., "Using Historical In-Process and Product Metrics for Early Estimation of Software Failures," in International Symposium on Software Reliability Engineering, 2006, pp. 62--74.

Digital Library

[12]

M. C. Ohlsson, von Mayrhauser, A., McGuire, B., Wohlin, C., "Code Decay Analysis of Legacy Software through Successive Releases," in IEEE Aerospace Conference, 1999, pp. 69--81.

[13]

T. J. Ostrand, Weyuker, E. J, Bell, R. M., "Where the Bugs Are," in the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), 2004, pp. 86--96.

Digital Library

[14]

R. Subramanyam, Krishnan, M. S., "Empirical Analysis of CK Metrics for Object-Oriented Design Complexity: Implications for Software Defects," IEEE Transactions on Software Engineering, vol. Vol. 29, pp. 297--310, 2003.

Digital Library

[15]

M.-H. Tang, Kao, M.-H., Chen, M.-H., "An empirical study on object-oriented metrics," in Sixth International Software Metrics Symposium, 1999, pp. 242--249.

Digital Library

[16]

M. A. Vouk, Tai, K. C., "Multi-Phase Coverage- and Risk-Based Software Reliability Modeling," in CASCON '93, 1993, pp. 513--523.

Digital Library

Cited By

Ahmad MGoseva-Popstojanova KLutz R(2024)The untold impact of learning approaches on software fault-proneness predictions: an analysis of temporal aspectsEmpirical Software Engineering10.1007/s10664-024-10454-829:4Online publication date: 8-Jun-2024
https://dl.acm.org/doi/10.1007/s10664-024-10454-8
Khil OYakovyna V(2023)Аналіз проблеми застосування методів машинного навчання для оцінювання та прогнозування дефектів програмного забезпеченняScientific Bulletin of UNFU10.36930/4033031633:3(110-116)Online publication date: 25-May-2023
https://doi.org/10.36930/40330316
Pal SSillitti A(2022)Cross-Project Defect Prediction: A Literature ReviewIEEE Access10.1109/ACCESS.2022.322118410(118697-118717)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3221184
Show More Cited By

Index Terms

Iterative identification of fault-prone binaries using in-process metrics
1. Software and its engineering
  1. Software creation and management
    1. Software development techniques
      1. Reusability
        Software product lines
    2. Software verification and validation
      1. Process validation

Recommendations

Fault prediction and the discriminative powers of connectivity-based object-oriented class cohesion metrics

Context: Several metrics have been proposed to measure the extent to which class members are related. Connectivity-based class cohesion metrics measure the degree of connectivity among the class members. Objective: We propose a new class cohesion metric ...
A comparison between software design and code metrics for the prediction of software fault content

Software metrics play an important role in measuring the quality of software. It is desirable to predict the quality of software as early as possible, and hence metrics have to be collected early as well. This raises a number of questions that has not ...
Interactive churn metrics: socio-technical variants of code churn

A central part of software quality is finding bugs. One method of finding bugs is by measuring important aspects of the software product and the development process. In recent history, researchers have discovered evidence of a "code churn" effect whereby ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEM '08: Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement

October 2008

374 pages

ISBN:9781595939715

DOI:10.1145/1414004

General Chair:
Dieter Rombach
University of Kaiserslautern and Fraunhofer Institute for Experimental Software Engineering (Germany)
,
Program Chairs:
Sebastian Elbaum
University of Nebraska-Lincoln (USA)
,
Jürgen Münch
Fraunhofer Institute for Experimental Software Engineering (Germany)

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 October 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEM '08

Sponsor:

ESEM '08: 2008 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement

October 9 - 10, 2008

Kaiserslautern, Germany

Acceptance Rates

Overall Acceptance Rate 130 of 594 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
409
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ahmad MGoseva-Popstojanova KLutz R(2024)The untold impact of learning approaches on software fault-proneness predictions: an analysis of temporal aspectsEmpirical Software Engineering10.1007/s10664-024-10454-829:4Online publication date: 8-Jun-2024
https://dl.acm.org/doi/10.1007/s10664-024-10454-8
Khil OYakovyna V(2023)Аналіз проблеми застосування методів машинного навчання для оцінювання та прогнозування дефектів програмного забезпеченняScientific Bulletin of UNFU10.36930/4033031633:3(110-116)Online publication date: 25-May-2023
https://doi.org/10.36930/40330316
Pal SSillitti A(2022)Cross-Project Defect Prediction: A Literature ReviewIEEE Access10.1109/ACCESS.2022.322118410(118697-118717)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3221184
Lee SWong WLi YChu W(2021)Software Fault-Proneness Analysis based on Composite Developer-Module NetworksIEEE Access10.1109/ACCESS.2021.31284389(155314-155334)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3128438
Maddila CBansal CNagappan NDumas MPfahl DApel SRusso A(2019)Predicting pull request completion time: a case study on large scale cloud servicesProceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3338906.3340457(874-882)Online publication date: 12-Aug-2019
https://dl.acm.org/doi/10.1145/3338906.3340457
Ramler RHimmelbauer JNatschläger T(2018)Building Defect Prediction Models in PracticeComputer Systems and Software Engineering10.4018/978-1-5225-3923-0.ch014(324-350)Online publication date: 2018
https://doi.org/10.4018/978-1-5225-3923-0.ch014
Illes-Seifert TPaech B(2018)Exploring the relationship of a file's history and its fault-pronenessInformation and Software Technology10.1016/j.infsof.2009.11.01052:5(539-558)Online publication date: 29-Dec-2018
https://dl.acm.org/doi/10.1016/j.infsof.2009.11.010
Madeyski LJureczko M(2018)Which process metrics can significantly improve defect prediction models? An empirical studySoftware Quality Journal10.1007/s11219-014-9241-723:3(393-422)Online publication date: 24-Dec-2018
https://dl.acm.org/doi/10.1007/s11219-014-9241-7
Ramler RHimmelbauer JNatschläger T(2014)Building Defect Prediction Models in PracticeHandbook of Research on Emerging Advancements and Technologies in Software Engineering10.4018/978-1-4666-6026-7.ch024(540-565)Online publication date: 2014
https://doi.org/10.4018/978-1-4666-6026-7.ch024
Hall TBeecham SBowes DGray DCounsell S(2012)A Systematic Literature Review on Fault Prediction Performance in Software EngineeringIEEE Transactions on Software Engineering10.1109/TSE.2011.10338:6(1276-1304)Online publication date: 1-Nov-2012
https://dl.acm.org/doi/10.1109/TSE.2011.103
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents