Using distinguishing tests to reduce the number of fault candidates



Mihai Nica Simona Nica Franz Wotawa


Tools for automated fault localization usually generate too many bug candidates depending on the underlying technique. Hence, further information is required in order to further restrict the bug candidates. Approaches that rely on specific knowledge of the program to be debugged like variable values at specific position in the source code, are not easily accessible for users especially in case of software maintenance. In order to avoid this problem we suggest to integrate testing for restricting the number of bug candidates. In particular, we suggest to compute possible corrections of the program and from this, distinguishing test cases. A distinguishing test case is a test that reveals different output values for two given program variants, given the same input values. Besides the formal definitions, and algorithms we present the first empirical results of our approach. The use of mutations and distinguishing test cases substantially reduces the number of bug candidates.

How to Cite

Nica, M. ., Nica, S. ., & Wotawa, F. . (2010). Using distinguishing tests to reduce the number of fault candidates. Annual Conference of the PHM Society, 2(2).
Abstract 1 | PDF Downloads 1




(A. J. Offutt and Zapf, 1996) G. Rothermel R. Untch A. J. Offutt, A. Lee and C. Zapf. An experimental determination of sufficient mutation operators. ACM Transactions on Software Engineering Methodology, 5:99–118, 1996.
(Abreu et al., 2009) Rui Abreu, Peter Zoeteweij, and Arjan J.C. van Gemund. Spectrum-based multiple fault localization. In Proc. IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 88–99, 2009.
(Binkley and Harman, 2004) David Binkley and Mark Harman. A survey of empirical results on program slicing. In Marvin Zelkowitz, editor, Advances in Software Engineering – Advances in Computers Vol. 62, pages 106–172. Academic Press Inc., 2004.
(Ceballos et al., 2006) R. Ceballos, R. M. Gasca, C. Del Valle, and D. Borrego. Diagnosing errors in dbc programs using constraint programming. Lecture Notes in Computer Science, 4177:200–210, 2006.
(Debroy and Wong, 2010 ) Vidroha Debroy and W. Eric Wong. Using mutation to automatically suggest fixes for faulty programs. In Third International Conference on Software Testing, Verification and Validation (ICST 2010). IEEE, 2010.
(Gent et al., 2006) I. P. Gent, C. Jefferson, and I. Miguel. Minion: A fast, scalable, constraint solver. 17th European Conference on Artificial Intelligence, ECAI-06, 2006.
(Jones and Harrold, 2005) J. A. Jones and M. J. Harrold. Empirical evaluation of the tarantula automatic fault-localization technique. In Proceedings, ASE’05, pages 273–282. ACM Press, 2005.
(Kusumoto et al., 2002) Shinji Kusumoto, Akira Nishimatsu, Keisuke Nishie, and Katsuro Inoue. Experimental evaluation of program slicing for fault localization. Empirical Software Engineering, 7:49–76, 2002.
(Mayer and Stumptner, 2003) W. Mayer and M. Stumptner. Model-based debugging using multiple abstract models. Proceedings of the 5th International Workshop on Automated and Algorithmic Debugging, AADEBUG-03:55–70, 2003.
(Mayer et al., 2009) Wolfgang Mayer, Rui Abreu, Markus Stumptner, and Arjan J.C. van Gemund. Prioritising model-based debugging diagnostic reports. In Proceedings of the International Workshop on Principles of Diagnosis (DX). 2009.
(Mayer, 2007) W. Mayer. Static and hybrid analysis in model-based debugging. PhD Thesis, School of Computer and Information Science, University of South Australia, 2007.
(Nica et al., 2009) M. Nica, J. Weber, and F. Wotawa. On the use of specification knowledge in program debugging. 20th International Workshop on Principles of Diagnosis, (DX-09), 2009.
(Reiter, 1987) Raymond Reiter. A theory of diagnosis from first principles. Artificial Intelligence, 32(1):57–95, 1987.
(Tretmans, 1996) J. Tretmans. Test generation with inputs, outputs and repetitive quiescence. Software - Concepts and Tools, 17(3):103–120, 1996.
(Weimer et al., 2009) Westley Weimer, Thanh Vu Nguyen, Claire Le Goues, and Stephanie Forrest. Automatically finding patches using genetic programming. In ACM/IEEE International Conference on Software Engineering (ICSE), pages 512–521, 2009.
(Wotawa and Nica, 2008) F. Wotawa and M. Nica. On the compilation of programs into their equivalent constraint representation. Informatika, 32:359–371, 2008.
(Wotawa et al., 2010) Franz Wotawa, Mihai Nica, and Bernhard K. Aichernig. Generating distinguish- ing tests using the minion constraint solver. In CSTVA 2010: Proceedings of the 2nd Workshop on Constraints for Testing, Verification and Analysis. IEEE, 2010.
(Yu-Seung Ma and Kwon., 2005 ) Jeff Offutt Yu- Seung Ma and Yong Rae Kwon. Mujava : An automated class mutation system. Software Testing, Verification and Reliability, 15:97–133, 2005.
(Zeller and Hildebrandt, 2002) Andreas Zeller and Ralf Hildebrandt. Simplifying and isolating failure- inducing input. IEEE Transactions on Software Engineering, 28(2), feb 2002.
(Zhang et al., 2005) Xiangyu Zhang, Haifeng He, Neelam Gupta, and Rajiv Gupta. Experimental evaluation of using dynamic slices for fault localization. In Sixth International Symposium on Automated & Analysis-Driven Debugging (AADEBUG), pages 33–42, 2005.
Poster Presentations