Comprehensive Evaluation of Matrix Factorization Models for Collaborative Filtering Recommender Systems.

Jesús Bobadilla; Jorge Dueñas Lerín; Fernando Ortega; Abraham Gutierrez

doi:10.9781/ijimai.2023.04.008

Authors

Jesús Bobadilla Universidad Politécnica de Madrid
Jorge Dueñas Lerín Universidad Politécnica de Madrid
Fernando Ortega Universidad Politécnica de Madrid
Abraham Gutierrez Universidad Politécnica de Madrid

DOI:

https://doi.org/10.9781/ijimai.2023.04.008

Keywords:

Collaborative Filtering, Matrix Factorization, Recommendation Systems

Supporting Agencies

This work has been co-funded by the Ministerio de Ciencia e Innovación of Spain and European Regional Development Fund (FEDER) under grants PID2019-106493RB-I00 (DL-CEMG) and the Comunidad de Madrid under Convenio Plurianual with the Universidad Politécnica de Madrid in the actuation line of Programa de Excelencia para el Profesorado Universitario

Abstract

Matrix factorization models are the core of current commercial collaborative filtering Recommender Systems. This paper tested six representative matrix factorization models, using four collaborative filtering datasets. Experiments have tested a variety of accuracy and beyond accuracy quality measures, including prediction, recommendation of ordered and unordered lists, novelty, and diversity. Results show each convenient matrix factorization model attending to their simplicity, the required prediction quality, the necessary recommendation quality, the desired recommendation novelty and diversity, the need to explain recommendations, the adequacy of assigning semantic interpretations to hidden factors, the advisability of recommending to groups of users, and the need to obtain reliability values. To ensure the reproducibility of the experiments, an open framework has been used, and the implementation code is provided.

Downloads

Download data is not yet available.

References

Z. Batmaz, A. Yurekli, A. Bilge, C. Kaleli, “A review on deep learning for recommender systems: challenges and remedies,” Artificial Intelligence Review, vol. 52, no. 1, pp. 1–37, 2019.

J. Bobadilla, S. Alonso, A. Hernando, “Deep learning architecture for collaborative filtering recommender systems,” Applied Sciences, vol. 10, no. 7, p. 2441, 2020.

B. Zhu, R. Hurtado, J. Bobadilla, F. Ortega, “An efficient recommender system method based on the numerical relevances and the non-numerical structures of the ratings,” IEEE Access, vol. 6, pp. 49935–49954, 2018.

J. Bobadilla, R. Lara-Cabrera, Á. González-Prieto, F. Ortega, “Deepfair: Deep learning for improving fairness in recommender systems,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 6, no. 6, pp. 86–94, 2021, doi: 10.9781/ijimai.2020.11.001.

J. Carbó, J. M. Molina, J. Dávila, “Fuzzy referral based cooperation in social networks of agents,” AI Communications, vol. 18, pp. 1–13, 2005. 1.

D. Medel, C. González-González, S. V. Aciar, “Social relations and methods in recommender systems: A systematic review,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 7, no. 4, p. 7, 2022, doi: 10.9781/ijimai.2021.12.004.

M. Caro-Martínez, G. Jiménez-Díaz, J. A. Recio- García, “Local modelagnostic explanations for black-box recommender systems using interaction graphs and link prediction techniques,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. InPress, no. InPress, p. 1, 2021, doi: 10.9781/ijimai.2021.12.001.

S. Afef, Z. Brahmi, M. Gammoudi, “Trust-based recommender systems: An overview,” in 27th IBIMA Conference, 05 2016.

I. Pinyol, J. Sabater-Mir, “Computational trust and reputation models for open multi-agent systems: a review,” Artificial Intelligence Review, vol. 40, pp. 1–25, Jun 2013, doi: 10.1007/s10462-011-9277-z.

Y. Deldjoo, M. Schedl, P. Cremonesi, G. Pasi, “Recommender systems leveraging multimedia content,” ACM Computing Surveys (CSUR), vol. 53, no. 5, pp. 1–38, 2020.

S. Kulkarni, S. F. Rodd, “Context aware recommendation systems: A review of the state of the art techniques,” Computer Science Review, vol. 37, p. 100255, 2020.

S. Forouzandeh, K. Berahmand, M. Rostami, “Presentation of a recommender system with ensemble learning and graph embedding: a case on movielens,” Multimedia Tools and Applications, vol. 80, no. 5, pp. 7805–7832, 2021.

R. Salakhutdinov, A. Mnih, “Probabilistic matrix factorization,” in Proceedings of the 20th International Conference on Neural Information Processing Systems, NIPS’07, Red Hook, NY, USA, 2007, p. 1257–1264, Curran Associates Inc.

Z. Wu, H. Tian, X. Zhu, S. Wang, “Optimization matrix factorization recommendation algorithm based on rating centrality,” in International Conference on Data Mining and Big Data, 2018, pp. 114–125, Springer.

C. Févotte, J. Idier, “Algorithms for nonnegative matrix factorization with the β-divergence,” Neural computation, vol. 23, no. 9, pp. 2421–2456, 2011.

F. Ortega, R. Lara-Cabrera, Á. González-Prieto, J. Bobadilla, “Providing reliability in recommender systems through bernoulli matrix factorization,” Information Sciences, vol. 553, pp. 110–128, 2021.

A. Hernando, J. Bobadilla, F. Ortega, “A non negative matrix factorization for collaborative filtering recommender systems based on a bayesian probabilistic model,” Knowledge-Based Systems, vol. 97, pp. 188–202, 2016.

B. M. Marlin, “Modeling user rating profiles for collaborative filtering,” Advances in neural information processing systems, vol. 16, 2003.

D. M. Blei, A. Y. Ng, M. I. Jordan, “Latent dirichlet allocation,” Journal of machine Learning research, vol. 3, no. Jan, pp. 993–1022, 2003.

T. Hofmann, “Learning what people (don’t) want,” in European Conference on Machine Learning, 2001, pp. 214– 225, Springer.

A. Gunawardana, G. Shani, “Evaluating recommender systems,” in Recommender systems handbook, Springer, 2015, pp. 265–308.

C. C. Aggarwal, “Evaluating recommender systems,” in Recommender systems, Springer, 2016, pp. 225–254.

J. Bobadilla, A. Gutiérrez, S. Alonso, Á. González- Prieto, “Neural collaborative filtering classification model to obtain prediction reliabilities,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 7, no. 4, pp. 18–26, 2022, doi: 10.9781/ijimai.2021.08.010.

S. Vargas, P. Castells, “Rank and relevance in novelty and diversity metrics for recommender systems,” in Proceedings of the fifth ACM conference on Recommender systems, 2011, pp. 109–116.

P. Castells, S. Vargas, J. Wang, “Novelty and diversity metrics for recommender systems: choice, discovery and relevance,” in Proceedings of the 33rd European Conference on Information Retrieval (ECIR’11), 2011.

S. Vargas, P. Castells, D. Vallet, “Intent-oriented diversity in recommender systems,” in Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval, 2011, pp. 1211– 1212.

F. M. Harper, J. A. Konstan, “The movielens datasets: History and context,” Acm transactions on interactive intelligent systems (tiis), vol. 5, no. 4, pp. 1–19, 2015, doi: https://doi.org/10.1145/2827872

J. Golbeck, J. A. Hendler, “Filmtrust: movie recommendations using trust in web-based social networks,” CCNC 2006. 2006 3rd IEEE Consumer Communications and Networking Conference, 2006., vol. 1, pp. 282–286, 2006, doi: 10.1109/CCNC.2006.1593032.

J. Miller, G. Southern, “Recommender system for animated video,” Issues in Information Systems, vol. 15, no. 2, pp. 321–7, 2014.

Y. Koren, R. Bell, C. Volinsky, “Matrix factorization techniques for recommender systems,” Computer, vol. 42, no. 8, pp. 30–37, 2009.

J. Bobadilla, A. Gutiérrez, S. Alonso, Á. González- Prieto, “Neural collaborative filtering classification model to obtain prediction reliabilities,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 7, no. 4, pp. 18–26, 2022, doi: 10.9781/ijimai.2021.08.010.

F. Ortega, B. Zhu, J. Bobadilla, A. Hernando, “Cf4j: Collaborative filtering for java,” Knowledge- Based Systems, vol. 152, pp. 94–99, 2018, doi: https://doi.org/10.1016/j.knosys.2018.04.008

F. Ortega, J. Mayor, D. López-Fernández, R. Lara- Cabrera, “Cf4j 2.0: Adapting collaborative filtering for java to new challenges of collaborative filtering based recommender systems,” Knowledge-Based Systems, vol. 215, p. 106629, 2021.