Using Combinatorial Benchmark Construction to Improve the Assessment of Concurrency Bug Detection Tools

by Jeremy S. Bradbury, Itai Segall, Eitan Farchi, Kevin Jalbert, David Kelk

Abstract

PADTAD 2012 PresentationMany different techniques for testing and analyzing concurrency programs have been proposed in the literature. Currently, it is difficult to assess the tness of a particular concurrency bug detection method and to compare it to other bug detection methods due to a lack of unbiased data that is representative of the kinds of concurrency programs that are used in practice. To address this problem we propose a new benchmark of concurrent Java programs that is constructed using combinatorial test design. In this paper we present our combinatorial model for creating a benchmark, we propose a new concurrency benchmark and we discuses the relationship between our new benchmarks and existing benchmarks. Specific combinations of the model parameters define different interleaving spaces, thus differentiating between different test tools.

Bibliographic Information [Bibtex format]

@inproceedings{BSF+12,
Author = {Jeremy S. Bradbury AND Itai Segall AND Eitan Farchi AND Kevin Jalbert AND David Kelk},
Title = {Using Combinatorial Benchmark Construction to Improve the Assessment of Concurrency Bug Detection Tools},
Booktitle = {Proc. of the 10th Workshop on Parallel and Distributed Systems: Testing, Analysis, and Debugging (PADTAD 2012)},
Month = {Jul.},
Year = {2012},
Pages = {25--35}
}

Paper: [PDF]   Presentation: [PDF]