Enhancing recommender systems accuracy by using user-items latent features similarity

Abstract

Various researches and experiments in the field of sociology have time and again proved that humans tend to associate and bond with people who are similar to them in some or the other aspects. This phenomenon is known as homophily . In the field of sales and marketing particularly for Ecommerce stores , Recommender system algorithms use this concept of homophily to predict the level of preference of a given item by a user based on the feedback on that item by other users and also on past purchases by same users. It is enhancing the customer loyalty by recommending him relevant products that he might not have actually looked across and hence increasing the crossselling of products. Recommender system algorithms are commonly based on content filtering, collaborative filtering or hybrid techniques. While the Content-based approaches make recommendations by considering the descriptions of the items already purchased by the user or his learned profile, Collaborative filtering takes in to account the feedback and reviews given by large numbers of users for various items and finds correlations / similarities among users . Hybrid approach combines content and collaborative methods. The employed techniques for Collaborative filtering includes probabilistic approaches, Bayesian networks, nearest neighbors algorithm; bio-inspired algorithms such as neural networks and genetic algorithms; fuzzy models, singular value decomposition techniques to reduce sparsity levels, etc. Memory-based Recommender system approaches the problem by using the entire database. Every time a prediction needs to be made while Model-based recommendation systems build a model based on the dataset of ratings. Thus after learning from dataset, the model is used to make recommendations without having to use the complete dataset every time. By using Latent Factor model hidden features about the relationship between user and items are learned. Singular Value Decomposition (SVD) transforms both items and users to the same latent factor space, thus making them directly comparable. In the proposed recommender system algorithm, we first use the cosine similarity to learn the latent elements that describe the inherent structure of relationship between items and users. Then for prediction of ratings of an unrated item by the active user we calculate similarity score between the user and unrated item, user and all other items rated by active user, based on learned latent features. Now k nearest neighbors to the unrated items are scanned based on calculated similarity score and their average is used to predict the rating for the unrated item. This novel method of prediction using user-item similarity based on learned latent features is in contrast with existing popular method of user-user similarity or, item-item similarity, and it shows significant gains as measured by various standard accuracy measures for recommender system algorithms as tested on MovieLens dataset (ml100k).