Abstract
A rule-based classifier is a type of classifier that classifies the samples only if the samples meet some rules, otherwise, it does not predict. Therefore, for a given dataset, the number of predictions is unknown. Contrary to what one might think, assessing the performance of rule-based classifiers is a challenging task. Currently, rule-based classifiers are behaved similar to the other classifiers and the fact that the number of predictions is unknown has not been taken into consideration. This motivates us to work on a new measure considering the information uncovered by the classifiers. In this paper we propose a new measure called Relative Information Superiority (RIS), that could be used in those problems where the number of predictions is unknown. In a comparison between RIS and accuracy, we show that RIS is more efficient when dealing with rule-based binary classification problems.