Abstract
Recommender Systems (RSs) are often assessed in off-line settings by measuring the system precision in predicting the observed user’s ratings or choices. But, when a precise RS is on-line, the generated recommendations can be perceived as marginally useful because lacking novelty. The underlying problem is that it is hard to build an RS that can correctly generalise, from the analysis of user’s observed behaviour, and can identify the essential characteristics of novel and yet relevant recommendations. In this paper we address the above mentioned issue by considering four RSs that try to excel on different target criteria: precision, relevance and novelty. Two state of the art RSs called SKNN and s-SKNN follow a classical Nearest Neighbour approach, while the other two, Q-BASE and Q-POP PUSH are based on Inverse Reinforcement Learning. SKNN and s-SKNN optimise precision, Q-BASE tries to identify the characteristics of POIs that make them relevant, and Q-POP PUSH, a novel RS here introduced, is similar to Q-BASE but it also tries to recommend popular POIs. In an off-line experiment we discover that the recommendations produced by SKNN and s-SKNN optimise precision essentially by recommending quite popular POIs. Q-POP PUSH can be tuned to achieve a desired level of precision at the cost of losing part of the best capability of Q-BASE to generate novel and yet relevant recommendations. In the on-line study we discover that the recommendations of SKNN and Q-POP PUSH are liked more than those produced by Q-BASE. The rationale of that was found in the large percentage of novel recommendations produced by Q-BASE, which are difficult to appreciate. However, Q-BASE excels in recommending items that are both novel and liked by the users.