Abstract
Software startups develop innovative, software-intensive product and services. Although the results of successful stories are ubiquitous in our daily life, like when we search online, use a social network, or book a hotel room, most of these companies fail. Many reasons could lead to failure, but research has shown that, experimentation, an approach based on testing business assumptions using experiments, is essential to improve these companies' success rate. Despite experimentation is at the core of methodologies well-known among practitioners, such as the Lean Startup, and , and the cost associated with these experiments were drastically reduced by the advent of cloud computing, several studies show that software startups do not use the approach often, focusing instead on fully developing an idea.
Although the reasons for this phenomenon are not fully understood, the lack of systematic techniques to operationalize experimentation in the context of software startups seems a key aspect.
This thesis aims to advance in this direction by proposing a technique for software startups to systematically identify hypotheses that will serve to guide the creation of experiments. This proposal is based on a better understanding of why these companies do not employ experimentation more often. To reach this goal, the thesis presents four empirical studies: a multiple-case study and a qualitative survey, to investigate the adoption and implementation of experimentation in software startups, a gray literature review, to describe how these companies currently handle hypotheses, and a design science research study, to develop an hypotheses elicitation technique. These studies led to several interesting results for experimentation and software startups. First, Hypotheses Engineering, an umbrella term for several activities to handle hypotheses, is defined based on empirical evidence. The XPro model explains experimentation adoption and implementation in software startups through a staged-process and identifies several inhibitors preventing these companies to experiment more often. Finally, HyMap is a novel technique, based on cognitive mapping, to suggest hypotheses to be tested using experiments by surfacing how founders understand their product and potential customers. The evaluation of HyMap in three software startups regarding utility, quality, and effectiveness led to promising results. This thesis represents several implications for theory and practice. The concept of Hypotheses Engineering is useful to guide the understanding of how the initial steps of experimentation are done and for the development of new techniques. Researchers, founders, and other startup stakeholders could use the better understanding of experimentation adoption and implementation to help these companies to reach better results or, at least, to reduce waste by failing faster. HyMap is a useful technique probably not only for startups but also for other development teams creating software in uncertain scenarios.