Model-based hypothesis testing of uncertain software systems
MetadataShow full item record
SubjectSoftware system testing; Formal methods; Markov Model; uncertainty analysis; INF/01; ING-INF/05
Nowadays, there exists an increasing demand for reliable software systems able to fulfill their requirements in different operational environments and to cope with uncertainty that can be introduced both at design-time and at runtime because of the lack of control over third-party system components and complex interactions among software, hardware infrastructures and physical phenomena. This article addresses the problem of the discrepancy between measured data at runtime and the design-time formal specification by using an inverse uncertainty quantification approach. Namely, we introduce a methodology called METRIC and its supporting toolchain to quantify and mitigate software system uncertainty during testing by combining (on-the-fly) model-based testing and Bayesian inference. Our approach connects probabilistic input/output conformance theory with statistical hypothesis testing in order to assess if the behaviour of the system under test corresponds to its probabilistic formal specification provided in terms of a Markov decision process. An uncertainty-aware model-based test case generation strategy is used as a means to collect evidence from software components affected by sources of uncertainty. Test results serve as input to a Bayesian inference process that updates beliefs on model parameters encoding uncertain quality attributes of the system under test. This article describes our approach from both theoretical and practical perspectives. An extensive empirical evaluation activity has been conducted in order to assess the cost-effectiveness of our approach. We show that, under same effort constraints, our uncertainty-aware testing strategy increases the accuracy of the uncertainty quantification process up to 50 times with respect to traditional model-based testing methods.
Showing items related by title, author, creator and subject.
Pahl C (Elsevier, 2007)Model-driven development is a software development framework that emphasises model-based abstraction and automated code generation. Service-based software architectures benefit in particular from semantic, ontology-based ...
An agile v-model for medical device software development to overcome the challenges with plan-driven software development lifecycles McHugh M; Cawley O; McCaffery F; Richardson I; Wang X (IEEE, 2013)Through the use of semi structured interviews with medical device software organizations it emerged that medical device software organizations are experiencing difficulties when following plan driven Software Development ...
Pahl C; Barrett R (2010)Service-oriented architecture is a recent conceptual framework for service-oriented software platforms. Architectures are of great importance for the evolution of software systems. We present a modelling and transformation ...