Abstract
Many recommender systems enrich traditional user-rating datasets with side information about features of items (e.g., genre, director, cast) and build user models as estimates of users’ interests on features. These models are usually evalu-ated based on their ability to discover items relevant to the users. However, as public datasets do not contain the ex-plicit opinions of users on features of items, user models are never evaluated in terms of matching between estimated and true preferences on features. In this paper we present on-going work aimed at filling this gap by using crowdsourcing to collect a dataset (PoliMovie) which contains the explicit preferences of users on both items and their attributes. We present some preliminary results based on the preferences collected from 341 users. The results confirm our initial in-tuition that traditional user models based on implicit user preferences on attributes do not match well with the explicit opinions of users on attributes: only 11% of the implicitly derived models are in agreement with the explicit opinion of users on attributes. These results are convincing enough to justify a much more extensive crowdsourced collection of data.