Evaluating defect prediction approaches: A benchmark and an extensive comparison
MetadataShow full item record
Reliably predicting software defects is one of the holy grails of software engineering. Researchers have devised and implemented a plethora of defect/bug prediction approaches varying in terms of accuracy, complexity and the input data they require. However, the absence of an established benchmark makes it hard, if not impossible, to compare approaches. We present a benchmark for defect prediction, in the form of a publicly available dataset consisting of several software systems, and provide an extensive comparison of well-known bug prediction approaches, together with novel approaches we devised. We evaluate the performance of the approaches using different performance indicators: classification of entities as defect-prone or not, ranking of the entities, with and without taking into account the effort to review an entity. We performed three sets of experiments aimed at (1) comparing the approaches across different systems, (2) testing whether the differences in performance are statistically significant, and (3) investigating the stability of approaches across different learners. Our results indicate that, while some approaches perform better than others in a statistically significant manner, external validity in defect prediction is still an open problem, as generalizing results to different contexts/learners proved to be a partially unsuccessful endeavor.
Showing items related by title, author, creator and subject.
Fronza, I; Phaphoom, N; Sillitti, A; Succi, G; Vlasenko, J; di Bella, E (Institute of Electrical and Electronics Engineers (IEEE), 2013)In the last decade, there has been increasing interest in pair programming (PP). However, despite the existing work, there is still a lack of substantial evidence of the effects of PP in industrial environments. To address ...
Weiler B; Gagliardi A; Lugli P (AMER CHEMICAL SOC, 2016)Anatase titanium dioxide is a highly promising material for memristors and photocatalysis. Multiple electronic transport processes are known to be influenced by defects in nanoscale anatase. Hence, in this study, we examine ...
Weiler B; Haeberle T; Gagliardi A; Lugli P (AMER INST PHYSICS, 2016)Ultrathin films of alumina were investigated by a compact kMC-model. Experimental jV-curves from Al/AlOx/Au-junctions with plasma-and thermal-grown AlOx were fitted by simulated ones. We found dominant defects at 2.3-2.5 ...