Project Members:
- Jeremy S. Bradbury
- Devin Kester
- Martin Mwebesa
- Kevin Jalbert
- James R. Cordy (Queen’s University)
- Juergen Dingel (Queen’s University)
Description:
The goal of this project is to explore the complementary relationship between testing and formal analysis with respect to debugging. We have developed an approach for the assessment of testing and formal analysis tools using metrics to measure the quantity and efficiency of each technique at finding bugs. Specifically we use our experimental mutation analysis framework (ExMAn) to symmetrically compare and evaluate tests versus properties using state-of-the art testing and analysis techniques and tools.
[Note: This project is closely related to the “Mutation Analysis of Concurrent Software” project. The techniques and tools developed in the mutation analysis project are utilized in the experimental comparison of bug detection tools.]
Publications:
- Devin Kester, Martin Mwebesa and Jeremy S. Bradbury. “How Good is Static Analysis at Finding Concurrency Bugs?”, Proc. of the 10th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2010), pages 115-124, Timisoara, Romania, Sept. 2010. Invited to a special journal issue.
- Jeremy S. Bradbury, James R. Cordy and Juergen Dingel. “Comparative Assessment of Testing and Model Checking Using Program Mutation”, In Proc. of the 3rd Workshop on Mutation Analysis (Mutation 2007), pages 210-219, Windsor, UK, Sept. 2007.
- Jeremy S. Bradbury. “Using Mutation for the Assessment and Optimization of Tests
and Properties”, Doctoral Symposium being held in conjunction with the International
Symposium on Software Testing and Analysis (ISSTA 2006), Portland Maine, USA,
July 2006, 4 pp. - Jeremy S. Bradbury, James R. Cordy and Juergen Dingel. “An Empirical Framework for Comparing Effectiveness of Testing and Property-Based Formal Analysis”, In Proc. of the 6th International ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering (PASTE 2005), pages 2-5, Lisbon, Portugal, Sept. 2005.
Posters and Demos:
- Jeremy S. Bradbury, James R. Cordy and Juergen Dingel. “Bugs and Concurrency: A Quantitative Assessment of Bug Detection Tools”, In Technology Showcase at the 16th Annual International Conference on Computer Science and Software Engineering (CASCON 2006), Markham, Ontario, Canada, Oct. 2006.