Testing Deep Learning Recommender Systems Models on Synthetic GAN-Generated Datasets.

Jesús Bobadilla; Abraham Gutiérrez

doi:10.9781/ijimai.2023.10.002

Authors

Jesús Bobadilla Universidad Politécnica de Madrid
Abraham Gutiérrez Universidad Politécnica de Madrid

DOI:

https://doi.org/10.9781/ijimai.2023.10.002

Keywords:

Collaborative Filtering, Deep Learning, Generative Adversarial Network, Synthetic Datasets, Recommendation Systems

Supporting Agencies

This work was partially supported by Ministerio de Ciencia e Innovación of Spain under the project PID2019-106493RB-I00 (DLCEMG) and the Comunidad de Madrid under Convenio Plurianual with the Universidad Politécnica de Madrid in the actuation line of Programa de Excelencia para el Profesorado Universitario.

Abstract

The published method Generative Adversarial Networks for Recommender Systems (GANRS) allows generating data sets for collaborative filtering recommendation systems. The GANRS source code is available along with a representative set of generated datasets. We have tested the GANRS method by creating multiple synthetic datasets from three different real datasets taken as a source. Experiments include variations in the number of users in the synthetic datasets, as well as a different number of samples. We have also selected six state-of-the-art collaborative filtering deep learning models to test both their comparative performance and the GANRS method. The results show a consistent behavior of the generated datasets compared to the source ones; particularly, in the obtained values and trends of the precision and recall quality measures. The tested deep learning models have also performed as expected on all synthetic datasets, making it possible to compare the results with those obtained from the real source data. Future work is proposed, including different cold start scenarios, unbalanced data, and demographic fairness.

Downloads

Download data is not yet available.

References

J. Bobadilla, A. González-Prieto, F. Ortega, R. Lara-Cabrera, “Deep learning feature selection to unhide demographic recommender systems factors,” Neural Computing and Applications, vol. 33, no. 12, pp. 7291-7308, 2021.

Y. Deldjoo, M. Schedl, P. Cremonesi, G. Pasi, “Recommender systems leveraging multimedia content,” ACM Computing Surveys (CSUR), vol. 53, no. 5, pp. 1-38, 2020.

S. Kulkarni, S.F. Rodd, “Context aware recommendation systems: A review of the state of the art techniques,” Computer Science Review, vol. 37, 100255, 2020.

J. Shokeen, C. Rana, “A study on features of social recommender systems”,Artificial Intelligence Review, vol. 53, no. 2, pp. 965-988, 2020.

J.B. Schafer, D. Frankowski, J. Herlocker, S. Sen, “Collaborative Filtering Recommender Systems,” in: The Adaptive Web. Lecture Notes in Computer Science, Brusilovsky, P., Kobsa, A., Nejdl, W. (eds), Springer, Berlin, Heidelberg, 2007, vol. 4321.

E. Cano, M. Morisio, “Hybrid recommender systems: A systematic literature review,” Intelligent Data Analysis, vol. 21, no. 6, pp. 1487-1524, 2017.

B. Zhu, R. Hurtado, J. Bobadilla, F. Ortega, “An efficient recommender system method based on the numerical relevances and the non-numerical structures of the ratings,” IEEE Access, vol. 6, pp. 49935-49954, 2018.

A. Mnih, R. R. Salakhutdinov, Probabilistic matrix factorization, Advances in neural information processing systems, vol. 20, 2007.

C. Févotte, J. Idier, “Algorithms for nonnegative matrix factorization with the β-divergence,” Neural computation, 2011, vol. 23, no. 9, pp. 2421-2456, 2011.

H.-J. Xue, X. Dai, J. Zhang, S. Huang, J. Chen, “Deep Matrix Factorization Models for Recommender Systems,” in: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, 2017, pp. 3203-3209.

J. Bobadilla, J. Dueñas, A. Gutiérrez, F. Ortega, “Deep Variational Embedding Representation on Neural Collaborative Filtering Recommender Systems,” Applied Sciences, vol. 12, no. 9, 4168, 2022.

X. He, L. Liao, H. Zhang, “Neural Collaborative Filtering,” International World Wide Web Conference Committee (IW3C2), 2017, pp. 173-182.

J. Bobadilla, F. Ortega, A. Gutiérrez, S. Alonso, “Classification-based Deep Neural Network Architecture for Collaborative Filtering Recommender Systems,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 6, no. 1, pp. 68-77, 2020.

J. Bobadilla, A. Gutiérrez, S. Alonso, A. González-Prieto, “Neural Collaborative Filtering Classification Model to Obtain Prediction Reliabilities,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 7, no. 4, pp. 18-26, 2022.

J. Bobadilla, A. Gutiérrez, R. Yera, L. Martínez “Creating Synthetic Datasets for Collaborative Filtering Recommender Systems using Generative Adversarial Networks,” Knowledge-Based Systems, pre-proof: 111016, 2023. https://doi.org/10.1016/j.knosys.2023.111016.

M. Gao, J. Zhang, J. Yu, J. Li, J. Wen, Q. Xiong, “Recommender systems based on generative adversarial networks: A problem-driven perspective,” Information Sciences, vol. 546, pp. 1166-118, 2021.

Y. Deldjoo; T. Noi, F.A. Merra, “A Survey on Adversarial Recommender Systems: From Attack/Defense Strategies to Generative Adversarial Networks,” ACM computing surveys, vol. 54, no. 2, pp. 1-38, 2021.

D.-K. Chae, J.-S. Kang, S.-W. Kim, J.-T. Lee, “CFGAN: a generic collaborative filtering framework based on generative adversarial networks,” in: Proceedings of the 27th, ACM International Conference on Information and Knowledge Management, CIKM 2018, 2018, pp. 137-146.

G. Guo, H. Zhou, B. Chen, et al., “IPGAN: Generating informative item pairs by adversarial sampling,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no.2, pp. 694-706, 2022.

H. Bharadhwaj, H. Park, B.Y. Lim “Recgan: recurrent generative adversarial networks for recommendation systems,” in: Proceedings of the 12th ACM, Conference on Recommender Systems, RecSys 2018, 2018, pp. 372-376.

J. Zhao, H. Li, L. Qu, Q. Zhang, Q. Sun, H. Huo, M. Gong, “DCFGAN: An adversarial deep reinforcement learning framework with improved negative sampling for session-based recommender systems,” Information sciences, vol. 596, pp. 222-235, 2022.

J. Wen, X. Zhu, C.D. Wang, Z. Tian, “A framework for personalized recommendation with conditional generative adversarial networks,” Knowledge and information systems, vol. 64, no. 10, pp. 2637-2660, 2022.

J. Sun, B. Liu, H. Ren, W. Huang, “NCGAN: A neural adversarial collaborative filtering for recommender system,” in: Journal of intelligent & fuzzy systems, vol. 42, no. 4, pp. 2915-2923, 2022.

W. Shafqat, Y.C. Byun, “A Hybrid GAN-Based Approach to Solve Imbalanced Data Problem in Recommendation Systems,” in: IEEE access, vol. 10, pp. 11036-11047, 2022.

M. Mladenov, C.W. Hsu, V. Jain, E. Ie, C. Colby, N. Mayoraz, H. Pham, D. Tran, I. Vendrov, C. Boutilier, “Demonstrating Principled Uncertainty Modeling for Recommender Ecosystems with RecSim NG,” in: RecSys 2020 - 14th ACM Conference on Recommender Systems, 2020, pp. 591–593.

J.C Shi, Y. Yu, Q. Da, S.Y. Chen, A.X. Zeng, “Virtual-Taobao: Virtualizing real-world online retail environment for reinforcement learning,” in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 4902–4909.

M. del Carmen, S. Ilarri, R. Hermos, R. Trillo-Lado, “Datagencars: A generator of synthetic data for the evaluation of contextaware recommendation systems,” Pervasive and Mobile Computing, vol. 38, pp. 516–541, 2017.

V. Provalov, E. Stavinova and P. Chunaev, “SynEvaRec: A Framework for Evaluating Recommender Systems on Synthetic Data Classes,” in: International Conference on Data Mining Workshops (ICDMW), Auckland, New Zealand, 2021, pp. 55-64.

F. Ortega, B. Zhu, J. Bobadilla, A. Hernando, “CF4J: Collaborative filtering for Java,” Knowledge-Based Systems, vol. 152, pp. 94-99, 2018.

M. Račinský, “MyAnimeList Dataset,” Kaggle, 2018. [Dataset]. Available: https://www.kaggle.com/azathoth42/myanimelist, doi: 10.34740/KAGGLE/DSV/45582.

F.M. Harper, J.A. Konstan, “The movielens datasets: History and context,” ACM Transactions on Interactive Intelligent Systems, vol. 5, no. 4, pp. 1-19, 2015.