Empirical Comparison of Bug Detection Techniques for Concurrent Software

Project Members:


The goal of this project is to explore the complementary relationship between testing and formal analysis with respect to debugging. We have developed an approach for the assessment of testing and formal analysis tools using metrics to measure the quantity and efficiency of each technique at finding bugs. Specifically we use our experimental mutation analysis framework (ExMAn) to symmetrically compare and evaluate tests versus properties using state-of-the art testing and analysis techniques and tools.

[Note: This project is closely related to the “Mutation Analysis of Concurrent Software” project. The techniques and tools developed in the mutation analysis project are utilized in the experimental comparison of bug detection tools.]


Posters and Demos:

  • Jeremy S. Bradbury, James R. Cordy and Juergen Dingel. “Bugs and Concurrency: A Quantitative Assessment of Bug Detection Tools”, In Technology Showcase at the 16th Annual International Conference on Computer Science and Software Engineering (CASCON 2006), Markham, Ontario, Canada, Oct. 2006.


  • ExMAn an Experimental Mutation Analysis Framework.
  • ConMAn (on Github) a set of Concurrency Mutation Analysis operators, are a plugin-in for the EXMan Framework.