Abstract
Collaborative filtering techniques have been studied extensively during the last decade. Many open source packages (Apache Mahout, LensKit, MyMediaLite, rrecsys etc.) have implemented them, but typically the top-N recommendation lists are only based on a highest predicted ratings approach. However, exploiting frequencies in the user/item neighborhood for the formation of the top-N recommendation lists has been shown to provide superior accuracy results in offline simulations. In addition, most open source packages use a time-independent evaluation protocol to test the quality of recommendations, which may result to misleading conclusions since it cannot simulate well the real-life systems, which are strongly related to the time dimension. In this paper, we have therefore implemented the time-aware evaluation protocol to the open source recommendation package for the R language — denoted rrecsys — and compare its performance across open source packages for reasons of replicability. Our experimental results clearly demonstrate that using the most frequent items in neighborhood approach significantly outperforms the highest predicted rating approach on three public datasets. Moreover, the time-aware evaluation protocol has been shown to be more adequate for capturing the life-time effectiveness of recommender systems.