Abstract
In today's ever increasing demand on loosely coupled systems, software providers need to produce solutions that are flexible enough to be configured at runtime, yet maintaining high quality. One way to address the quality of such systems is through testing. Software testing provides software providers and their clients with techniques to characterize the internal and external quality of their systems, by specifying test cases to check how systems react under different circumstances. In the context of web services the situation changes, software providers become service providers. In other words, they do not provide a physical software to their clients, instead they allow them to invoke remote code from their system. This change affects the way we look at testing itself. In the web service paradigm clients do not own services, instead they invoke them. Different approaches for testing web services have been reported in the literature, with different techniques and using different input information to generate test cases. Considering the topic of web services testing, as a subject for this systematic review is a challenging undertaking, mainly due to the extensive amount of literature in the subject. To narrow it, we have focused on one single particularity that affects dramatically the testing process. That is the specifications used to generate test cases. To this end, in this paper we present a systematic review of the specifications used for testing web services. The outcome of this study answers key questions involved in the advantages and disadvantages of the existing specifications as well as the potential improvements that could lead to more robust web services testing process.