Qualitative and Quantitative Assessment of Integration Testing for Model-Based Software in the Automotive Industry

Abstract: Background: Integration testing of vehicle software in the automotive industry relies heavily on simulation models. As they replicate actual vehicle functions in the testing process, they increase in size and amount of interconnectivity as rapidly as the actual functions. Valid simulation models are a precondition for valid integration testing. Hence, assessment of the models is of high importance in industry. At the same time, assessment approaches for model-based software validated in industry are scarce. Objective: The goal of this thesis is to assess current integration testing in the automotive industry and extend the validation of simulation models. Accordingly, we aim to collect insights from the practitioners in the field, including elicitation of actual challenges in the industry, state-of-the-practice processes, and assessments of applicability for validation approaches. Method: To achieve the objectives we combine quantitative and qualitative research methods including interviews, workshops, surveys, literature reviews, software measurements, correlation analysis, and statistical tests. In five studies attached with this thesis we combine multiple research methods to achieve high validity and to ensure all presented approaches are applicable in industry. Results: We elicited and categorized challenges from practitioners in practice, particularly in the field of integration testing for automotive software and analyze the current software development process. We present measurement results from complexity and size metrics, as a first assessment of the models. In addition to single measurements, we show how to evaluate software measurement results collected over time and how they can be related to model quality. We show that outlier analysis can help detecting impactful observations in the model development process. Furthermore, we found five approaches for the prediction of software model growth data and elaborate on their strengths and weaknesses, in practice. Next to providing actual approaches, we present practitioners expectations towards maintainability measurements and measurement predictions. Conclusion: In this work we contribute to the understanding of concrete challenges in industry, we describe current processes, and provide approaches applicable in industry to address elicited challenges. With our work we improve the current assessment of validity of simulation models in integration testing in the automotive industry.