Test Models and Algorithms for Model-Based Testing of Software Product Lines

Abstract: Software product line (SPL) engineering has become common practice for mass production and customization of software. A software product line comprises a family of software systems which share a managed core set of artifacts. There are also a set of well-defined variabilities between the products of a product line. The main idea in SPL engineering is to enable systematic reuse in different phases of software development to reduce cost and time to release.Model-Based Testing (MBT) is a technique that is widely used for checking the quality of software systems. In MBT, test cases are generated from an abstract model, which captures the desired behavior of the system. Then, the test cases are executed against a real implementation of the system and the compliance of the implementation to the specification is checked by comparing the observed outputs with the ones prescribed by the model.Software product lines have been applied in many domains in which sys- tems are mission critical and MBT is one of the techniques that is widely used for quality assurance of such systems. As the number of products can be potentially large in an SPL, using conventional approaches for MBT of the products of an SPL individually and as single systems can be very costly and time consuming. Hence, several approaches have been proposed in order to enable systematic reuse in different phases of the MBT process.An efficient modeling technique is the first step towards an efficient MBT technique for SPLs. There have been several formalisms proposed for modeling SPLs. In this thesis, we conduct a study on such modeling techniques, focusing on three fundamental formalisms, namely featured transition systems, modal transition systems, and product line calculus of communicating systems. We compare the expressive power and the succinctness of these formalisms.Furthermore, we investigate adapting existing MBT methods for efficient testing of SPLs. As a part of this line of our research, we adapt the test case generation algorithm of one of the well-known black-box testing approaches, namely, Harmonized State Identification (HSI) method by exploiting the idea of delta-oriented programming. We apply the adapted test case generation algorithm to a case study taken from industry and the results show up to 50 percent reduction of time in test case generation by using the delta-oriented HSI method.In line with our research on investigating existing MBT techniques, we compare the relative efficiency and effectiveness of the test case generation algorithms of the well-known Input-Output Conformance (ioco) testing approach and the complete ioco which is another testing technique used for input output transition systems that guarantees fault coverage. The comparison is done using three case studies taken from the automotive and railway domains. The obtained results show that complete ioco is more efficient in detecting deep faults (i.e., the faults reached through longer traces) in large state spaces while ioco is more efficient in detecting shallow faults (i.e., the faults reached through shorter traces) in small state spaces.