ScoredKNN: An Efficient KNN Recommender Based on Dimensionality Reduction for Big Data

E-commerce companies have an inevitable need in employing recommender systems in order to enhance the user experience, increase customer satisfaction, and drive sales. One of the most popular, intuitive and explainable recommender algorithm is the K-nearest neighbors (KNN) algorithm which is a well-known non-parametric collaborative filtering (CF) method. However, when dealing with big data, applying KNN poses computational challenges in terms of both time and space consumption. Several solutions proposed, but none of them could become a standard solution up to now. To address this issue, we propose a dimension reduction based approach with scoring functions which is applicable on all neighboring methods. With the help of this approach, similarity calculation is reduced into one dimension instead of two dimensions. The proposed approach reduces the KNN complexity from O(n²) to O(n) and it has been evaluated on both publicly available datasets and also real-world e-commerce datasets of an e-commerce services provider company Frizbit S.L.. We have compared our method with state-of-the-art recommender systems algorithms and evaluated based on the criteria: time consumption, space consumption and accuracy. According to the experimental results, we have observed that our proposed approach ScoredKNN achieves a pretty good accuracy (in terms of MAE) and lower time/space costs.