Abstract
Recommender Systems were created to support users in situations of information overload. However, users are consciously or unconsciously influenced by several factors in their decision-making. We analysed a historical dataset from a meta-search booking platform with the aim of exploring how these factors influence user choices in the context of online hotel search and booking. Specifically, we focused our study on the influence of (i) ranking position, (ii) number of reviews, (iii) average ratings and (iv) price when analysing users’ click behaviour. Our results confirmed conventional wisdom that position and price were the “two elephants in the room” heavily influencing user decision-making. Thus, they need to be taken into account when, for instance, trying to learn user preferences from clickstream data. Using the results coming from this analysis, we performed an online A/B test on this meta-search booking platform comparing the current policy with a price-based re-rank policy. Our online experiments suggested that, although in offline experiments items with lower prices tend to have a higher Click-Through Rate, in an online context a price-based re-rank was only capable to improve the Click-Through Rate metric for the first positions of the recommended lists.