Masters Thesis

Thesis submission: An Empirical Investigation on the Application of Statistical Estimators in Mutation Testing

Abstract

Mutation analysis is the gold standard to measure the quality of test suites. It involves the evaluation of software test suites on detecting seeded faults (mutations). A key concern in mutation analysis is the question of equivalent mutants. These mutants, while syntactically different from original, are semantically indistinguishable from the original program. The presence of an unknown number of equivalent mutants affects the reliability of mutation score as test adequacy metric because it is unclear what percentage of mutants can be detected from a set of not killed mutants. Numerous researchers have tried to mitigate the issue by eliminating them, and these measures have not borne fruit.

This thesis proposes a new solution: Here, the number of killable mutants are estimated using statistical measures, which are used to compute an approximation of the ``real’’ mutation score with statistical confidence. The results demonstrated that the statistical estimates obtained from the Ichaos estimator are closer than the Choas estimator to the manually classified ground truth estimates. Subsequently, the mutation score results obtained from Ichaos and Chaos estimates were seen overlapping and closer to the ground truth mutation score. The results highlight the usefulness of the Ichoas estimator to enhance the mutation score reliability. The accuracy of the results is limited since the traditional/generated suites are inadequate.

Co-advised with Alessio Gambi, under Prof. Gordon Fraser