Open Access

ARTICLE

Effectiveness Assessment of the Search-Based Statistical Structural Testing

Yang Shi*, Xiaoyu Song, Marek Perkowski, Fu Li
Department of Electrical and Computer Engineering, Portland State University, Portland, 97201, USA
* Corresponding Author: Yang Shi. Email:

Computers, Materials & Continua 2022, 70(2), 2191-2207. https://doi.org/10.32604/cmc.2022.018718

Received 19 March 2021; Accepted 26 April 2021; Issue published 27 September 2021

Abstract

Search-based statistical structural testing (SBSST) is a promising technique that uses automated search to construct input distributions for statistical structural testing. It has been proved that a simple search algorithm, for example, the hill-climber is able to optimize an input distribution. However, due to the noisy fitness estimation of the minimum triggering probability among all cover elements (Tri-Low-Bound), the existing approach does not show a satisfactory efficiency. Constructing input distributions to satisfy the Tri-Low-Bound criterion requires an extensive computation time. Tri-Low-Bound is considered a strong criterion, and it is demonstrated to sustain a high fault-detecting ability. This article tries to answer the following question: if we use a relaxed constraint that significantly reduces the time consumption on search, can the optimized input distribution still be effective in fault-detecting ability? In this article, we propose a type of criterion called fairness-enhanced-sum-of-triggering-probability (p-L1-Max). The criterion utilizes the sum of triggering probabilities as the fitness value and leverages a parameter p to adjust the uniformness of test data generation. We conducted extensive experiments to compare the computation time and the fault-detecting ability between the two criteria. The result shows that the 1.0-L1-Max criterion has the highest efficiency, and it is more practical to use than the Tri-Low-Bound criterion. To measure a criterion’s fault-detecting ability, we introduce a definition of expected faults found in the effective test set size region. To measure the effective test set size region, we present a theoretical analysis of the expected faults found with respect to various test set sizes and use the uniform distribution as a baseline to derive the effective test set size region’s definition.

Keywords


Cite This Article

Y. Shi, X. Song, M. Perkowski and F. Li, "Effectiveness assessment of the search-based statistical structural testing," Computers, Materials & Continua, vol. 70, no.2, pp. 2191–2207, 2022.



This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 1169

    View

  • 595

    Download

  • 0

    Like

Share Link

WeChat scan