Attentive Flexible Translation Embedding in Top-N Sparse Sequential Recommendations.

Min Ji Seo; Myung Ho Kim

doi:10.9781/ijimai.2022.10.007

Authors

Min Ji Seo Soongsil University
Myung Ho Kim Soongsil University

DOI:

https://doi.org/10.9781/ijimai.2022.10.007

Keywords:

Deep Learning, Gated Graph Neural Network, Knowledge Graph Embedding, Recommendation Systems, Self-Attention, Sequential Recommendation

Supporting Agencies

This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the Innovative Human Resource Development for Local Intellectualization support program (IITP-2022- RS-2022-00156360) supervised by the IITP (Institute for Information & communications Technology Planning & Evaluation).

Abstract

Sequential recommendation aims to predict the user’s next action based on personal action sequences. The major challenge in this task is how to achieve high performance recommendation under the data sparsity problem. Translation-based recommendations, which learn distance metrics to capture interactions between users and items in sequential recommendations, are a promising method to overcome this issue. However, a disadvantage of translation-based recommendations is that they capture long-term preferences of the user and complex item transitions. In this paper, we propose attentive flexible translation for recommendations (AFTRec) to tackle data sparsity problem by capturing a user’s dynamic preferences and complex interactions between items in user’s purchasing behaviors. In particular, we first encode semantic information of an item related to user’s purchasing behaviors as the user-specific item translation vectors. We also design a transition graph and encode complex item transitions as correlation-specific item translation vectors. Finally, we adopt a flexible distance metric that considers directions with respect to the translation vectors in the same space for predicting the next item. To evaluate the performance of our method, we conducted experiments on four sparse datasets and one dense dataset with different domains. The experimental results demonstrate that our proposed AFTRec outperforms the state-of-the-art baselines in terms of normalized discounted cumulative
gain and hit rate on sparse datasets.

Downloads

Download data is not yet available.

References

C. Feng, J. Liang, P. Song, and Z. Wang, “A fusion collaborative filtering method on sparse data in recommender systems,” Information Sciences, vol. 421, pp. 365-379, 2020, doi: 10.1016/j.ins.2020.02.052.

J. Bobadilla, S. Alonso, and A. Hernando, “Deep learning architecture for collaborative filtering recommender systems,” Applied Sciences, vol. 10, no. 7, pp. 2441, 2020, doi:10.3390/app10072441.

A. Gazdar, and L. Hidri, “A new similarity measure for collaborative filtering based recommender systems,” Knowledge-Based Systems, vol. 188, 2020, doi: 10.1016/j.knosys.2019.105058.

Y. Koren, R. Bell, and Chris Volinsky, “Matrix factorization techniques for recommender systems,” Computer, vol. 42, no. 8 pp. 30-37, 2009, doi: 10.1109/MC.2009.263.

D. Jannach, M. Ludewig, and L. Lerche, “Session-based item recommendation in e-commerce: on short-term intents, reminders, trends and discounts,” User Modeling and User-Adapted Interaction, vol. 27, pp.351-392, 2017, doi: 10.1007/s11257-017-9194-1.

A. Luo, P. Zhao, Y. Liu, F. Zhuang, D. Wang, J. Xu, et al., “Collaborative Self-Attention Network for Session-based Recommendation,” in Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, Yokohama, Japan, 2020, pp.2591-2597.

S. Sun, Y. Tang, Z. Dai, and F. Zhou, “Self-Attention Network for Session-Based Recommendation with Streaming Data Input,” IEEE Access, vol. 7, pp. 110499-110509, 2019, doi: 10.1109/ACCESS.2019.2931945.

D. Hu, L. Wei, W. Zhou, X. Huai, Z, Fang, and S. Hu, “PEN4Rec: Preference Evolution Networks for Session-based Recommendation,” arXiv preprint arXiv:2106.09306, 2021.

S. Rendle, C. Freudenthaler, and L. Schmidt-Thieme, “Factorizing Personalized Markov Chains for Next-basket Recommendation,” in Proceedings of the 19th International Conference on World Wide Web, North Carolina, USA, 2010, pp. 811–820.

D. W. Hogg, and D. Foreman-Mackey, “Data analysis recipes: Using markov chain monte carlo,” The Astrophysical Journal Supplement Series, vol. 236, no.1, pp.11, 2018, doi: 10.3847/1538-4365/aab76e.

P. Tengkiattrakul, S. Maneeroj, and A. Takasu, “Attentive Hybrid Collaborative Filtering for Rating Conversion in Recommender Systems,” in Proceedings of the International Conference on Energy, Water and Environment, Venice, Italy, 2021, pp.151-165.

Y. Zhang, Y. He, J. Wang, and J. Caverlee, “Adaptive Hierarchical Translation-based Sequential Recommendation,” in Proceedings of the Web Conference 2020, Taipei, Taiwan, 2020, pp. 2984-2990.

Y. Ding, Y. Ma, W. Wong, and T. S. Chua, “Modeling Instant User Intent and Content-level Transition for Sequential Fashion Recommendation,” IEEE Transactions on Multimedia, preprint.

A. Garcia-Duran, R. Gonzalez, D. Onoro-Rubio, M. Niepert, and H. Li, “Transrev: Modeling reviews as translations from users to items,” Advances in Information Retrieval, vol.12035, pp.234-248, 2020. doi: 10.1007/978-3-030-45439-5_16.

Q. Wang, Z. Mao, B. Wang, and L. Guo, “Knowledge graph embedding: A survey of approaches and applications,” IEEE Transactions on Knowledge and Data Engineering, vol. 29, no. 12, 2017, pp. 2724-2743, DOI: 10.1109/TKDE.2017.2754499.

M. Brandalero, M. Shafique, L. Carro, and A. C. S. Beck, “Transrec: Improving adaptability in single-ISA heterogeneous systems with transparent and reconfigurable acceleration,” in 2019 Design, Automation & Test in Europe Conference & Exhibition, Florence, Italy, 2019, pp. 582-585.

Y. Tay, L. Anh-Tuan, and S. C. Hui, “Latent relational metric learning via memory-based attention for collaborative ranking,” in Proceedings of the 2018 World Wide Web Conference, Lyon, France, 2018, pp. 729-739.

C. K. Hsieh, L. Yang, Y. Cui, T. Y. Lin, S. Belongie, and D. Estrin, “Collaborative metric learning,” in Proceedings of the 26th international conference on world wide web, Geneva, Switzerland, 2017, pp. 193-201.

A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, and O. Yakhnenko, “Translating embeddings for modeling multirelational data,” in Advances in Neural Information Processing Systems, Lake Tahoe, Nevada, 2013, pp. 2787–2795.

Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel, “Gated graph sequence neural networks,” arXiv preprint arXiv:1511.05493, 2015.

J. Feng, M. Huang, M. Wang, M. Zhou, Y. Hao and X. Zhu, “Knowledge graph embedding by flexible translation,” in Fifteenth International Conference on the Principles of Knowledge Representation and Reasoning, Cape Town, South Africa, 2017.

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, et al., “Attention is all you need,” In Advances in neural information processing systems, California, USA, 2017, pp. 5998-6008.

J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” arXiv preprint arXiv:1412.3555, 2014.

S. Kabbur, X. Ning, and G. Karypis, “Fism: factored item similarity models for top-n recommender systems,” in Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, Chicago, USA, 2013, pp. 659-667.

X. He, Z. He, J. Song, Z. Liu, Y. G. Jiang, and T. S. Chua, “Nais: Neural attentive item similarity model for recommendation,” IEEE Transactions on Knowledge and Data Engineering, vol. 30, no. 12, 2018, pp. 2354-2366, doi: 10.1109/TKDE.2018.2831682.

J. Bobadilla, A. Gutiérrez, S. Alonso and A. González-Prieto, “Neural Collaborative Filtering Classification Model to Obtain Prediction Reliabilities,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 7, no. 4, 2022, pp. 18-26, doi: 10.9781/ijimai.2021.08.010.

J. Bobadilla, R. Lara-Cabrera, A. González-Prieto and F. Ortega, “DeepFair: Deep Learning for Improving Fairness in Recommender Systems,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 6, no. 6, 2021, pp. 86-94, doi: 10.9781/ijimai.2020.11.001.

B. Hidasi, and A. Karatzoglou, “Recurrent neural networks with top-k gains for session-based recommendations,” in Proceedings of the Conference on Information and Knowledge Management, Turin, Italy, 2018, pp. 843–852.

P. M. Gabriel De Souza, D. Jannach, and A. M. Da Cunha, “Contextual hybrid session-based news recommendation with recurrent neural networks,” IEEE Access, vol. 7, 2019, pp.169185-169203, doi: 10.1109/ACCESS.2019.2954957.

S. Sun, Y. Tang, Z. Dai, and F. Zhou, “Self-attention network for sessionbased recommendation with streaming data input,” IEEE Access, vol. 7, 2019, pp. 110499–110599, doi: 10.1109/ACCESS.2019.2931945.

C. Xu, J. Feng, P. Zhao, F. Zhuang, D. Wang, Y. Liu, et al., “Long- and short-term self-attention network for sequential recommendation,” Neurocomputing, vol. 423, 2021, pp. 580-589, doi: 10.1016/j.neucom.2020.10.066.

J. Li, P. Ren, Z. Chen, Z. Ren, T. Lian, and J. Ma, “Neural attentive session-based recommendation,” in Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore, 2017, pp. 1419-1428.

W. C. Kang, and J. McAuley, “Self-attentive sequential recommendation,” in 2018 IEEE International Conference on Data Mining, Singapore, 2018, pp. 197-206.

L. Wu, S. Li, C. J. Hsieh, and J. Sharpnack, “SSE-PT: Sequential recommendation via personalized transformer,” in Fourteenth ACM Conference on Recommender Systems, Brazil, 2020, pp. 328-337.

J. Li, Y. Wang, and J. McAuley, “Time interval aware self-attention for sequential recommendation,” in Proceedings of the 13th international conference on web search and data mining, Texas, USA, 2020, pp. 322-330.

J. Tang, and K. Wang, “Personalized top-n sequential recommendation via convolutional sequence embedding,” in Proceedings of the ACM Conference on Web Search and Data Mining, California, USA, 2018, pp. 565–573.

F. Yuan, A. Karatzoglou, I. Arapakis, J. M. Jose, and X. He, “A simple convolutional generative network for next item recommendation,” in Proceedings of the ACM Conference on Web Search and Data Mining, Melbourne, Australia, 2019. pp. 582–590.

W. C. Kang, M. Wan, and J. McAuley, “Recommendation Through Mixtures of Heterogeneous Item Relationships,” in Proceedings of ACM Conference on Information and Knowledge Management, Turin, Italy, 2018, pp. 1143-1152.

B. Wu, X. He, Z. Sun, L. Chen, and Y. Ye, “ATM: An attentive translation model for next-item recommendation,” IEEE Transactions on Industrial Informatics, vol. 16, no. 3, 2020, pp. 1448-1459, doi: 10.1109/TII.2019.2947174.

R. Pasricha, and J. Mcauley, “Translation-based factorization machines for sequential recommendation,” in Proceedings of the 12th ACM Conference on Recommender Systems, Vancouver, Canada, 2018, pp.63-71.

Y. Zhan, Y. He, J. Wang, and J. Caverlee, “Adoptive hierarchical translation-based sequential recommendation,” in Proceedings of the Web Conference 2020, Taipei, Taiwan, 2020, pp. 2984-2990.

F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” IEEE transactions on neural networks, vol. 20, no. 1, 2009, pp.61-80, doi: 10.1109/TNN.2008.2005605.

C. Ma, P. Kang, and X. Liu, “Hierarchical gating networks for sequential recommendation,” in Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, Alaska, USA, 2019, pp. 825-833.

Y. Lin, Z. Liu, M. Sun, Y. Liu, and X. Zhu, “Learning entity and relation embeddings for knowledge graph completion,” in Proceedings of 29th AAAI conference on artificial intelligence, Texas, USA, 2015, pp. 2181– 2187.

T. Lacroix, N. Usunier, and G. Obozinski, “Canonical tensor decomposition for knowledge base completion,” in Proceedings of the International Conference on Machine Learning, Stockholm, Sweden, 2018, pp. 2863-2872.

D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473, 2014.

T. Dettmers, P. Minervini, P. Stenetorp, and S. Riedel, “Convolutional 2d knowledge graph embeddings,” in Thirty-second AAAI conference on artificial intelligence, Louisiana, USA, 2018.

Z. Sun, Z. H. Deng, J. Y. Nie, and J. Tang, “RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space,” in International Conference on Learning Representations, Louisiana, USA, 2018.

S. M. Kazemi, and D. Poole, “Simple embedding for link prediction in knowledge graphs,” in The Thirty-second Annual Conference on Neural Information Processing Systems, Montreal, Canada, 2018.

D. P. Kingma, and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.

J. J. McAuley, C. Targett, Q. Shi, and A. van den Hengel, “Image-based recommendations on styles and substitutes,” in Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, Santiago, Chile, 2015, pp. 43-52.