Source Credibility Assessment in the Realm of Information Disorder: A Literature Review

Alessia Cosentino; Carmen De Maio; Domenico Furno; Mariacristina Gallo; Vincenzo Loia

doi:10.9781/ijimai.2025.01.002

Authors

Alessia Cosentino University of Salerno Fisciano (Italy)
Carmen De Maio University of Salerno Fisciano (Italy)
Domenico Furno University of Salerno Fisciano (Italy)
Mariacristina Gallo University of Salerno Fisciano (Italy)
Vincenzo Loia University of Salerno Fisciano (Italy)

DOI:

https://doi.org/10.9781/ijimai.2025.01.002

Keywords:

Credibility Assessment, Information Disorder, Reliability, Source

Supporting Agencies

This work was partially supported by project SERICS (PE00000014) under the MUR National Recovery and Resilience Plan funded by the European Union -NextGenerationEU.

Abstract

The proliferation of information disorder in the digital age has sparked a growing concern regarding the credibility of sources disseminating information. This review examines the evolving landscape of source credibility within information disorder. The review synthesizes key findings and trends related to the factors influencing source credibility, including available tools, shared indicators, and existing methods experimented with in calculating source credibility. The analysis highlights that from a more commercial point of view, several tools are aimed at analyzing the content’s credibility and studying the sources’ credibility. However, from a methodological point of view, there is still something more to do. Indicators that can be used to carry out a source credibility assessment focus on the structure and design of the source, excluding others indicating how the page traffic could be. As for the techniques to be used to assess the credibility of a source, it emerged that more innovative techniques, such as deep-learning, are being developed alongside slightly more classical statistical methods. The review analyzes 23 papers from Conferences and 22 from Journals published in recent years. It also identifies avenues for future inquiry and the development of effective strategies to combat the challenges posed by misinformation in the digital era.

Downloads

Download data is not yet available.

References

D. Zhang, D. Wang, N. Vance, Y. Zhang, S. Mike, “On scalable and robust truth discovery in big data social media sensing applications,” IEEE Transactions on Big Data, vol. 5, no. 2, pp. 195–208, 2018.

H. Elmimouni, A. Forte, J. Morgan, “Why people trust wikipedia articles: Credibility assessment strategies used by readers,” in Proceedings of the 18th International Symposium on Open Collaboration, 2022, pp. 1–10.

S. Lee, J. Kim, D. Kim, K. J. Kim, E. Park, “Computational approaches to developing the implicit media bias dataset: Assessing political orientations of nonpolitical news articles,” Applied Mathematics and Computation, vol. 458, p. 128219, 2023.

A. A. Shah, S. D. Ravana, S. Hamid, M. A. Ismail, “Web pages credibility scores for improving accuracy of answers in web-based question answering systems,” IEEE Access, vol. 8, pp. 141456–141471, 2020.

X. Zhou, R. Zafarani, “A survey of fake news: Fundamental theories, detection methods, and opportunities,” ACM Computing Surveys (CSUR), vol. 53, no. 5, pp. 1–40, 2020.

B. Umeogu, “Source credibility: A philosophical analysis,” Open Journal of Philosophy, vol. 2, no. 2, pp. 112–115, 2012.

P. Meel, D. K. Vishwakarma, “Fake news, rumor, information pollution in social media and web: A contemporary survey of state-of-the-arts, challenges and opportunities,” Expert Systems with Applications, vol. 153, p. 112986, 2020.

E. Saquete, D. Tomás, P. Moreda, P. Martínez- Barco, M. Palomar, “Fighting post-truth using natural language processing: A review and open challenges,” Expert Systems with Applications, vol. 141, p. 112943, 2020.

S. M. Shariff, “A review on credibility perception of online information,” in 2020 14th International Conference on Ubiquitous Information Management and Communication (IMCOM), 2020, pp. 1–7, IEEE.

M. Faraon, A. Jaff, L. P. Nepomuceno, V. Villavicencio, “Fake news and aggregated credibility: Conceptualizing a co-creative medium for evaluation of sources online,” International Journal of Ambient Computing and Intelligence (IJACI), vol. 11, no. 4, pp. 93–117, 2020.

M. Mensio, H. Alani, “Misinfome: Who’s interacting with misinformation?,” in Proceedings of the 18th International Semantic Web Conference (ISWC 2019): Posters & Demonstrations, Industry and Outrageous Ideas Tracks, CEUR WS, October 2019, CEUR Workshop Proceedings.

M. Fernandez, H. Alani, “Online misinformation: Challenges and future directions,” in Companion Proceedings of the Web Conference 2018, 2018, pp. 595–602.

J. Park, R. D. Ellezhuthil, J. Isaac, C. Mergerson, L. Feldman, V. Singh, “Misinformation detection algorithms and fairness across political ideologies: The impact of article level labeling,” in Proceedings of the 15th ACM Web Science Conference 2023, 2023, pp. 107–116.

Y. Jo, M. Kim, K. Han, “How do humans assess the credibility on web blogs: Qualifying and verifying human factors with machine learning,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 2019, pp. 1–12.

C. Soohoo, D. R. Danielson, L. Marable, J. Stanford, E. R. Tauber, “How do users evaluate the credibility of web sites? a study with over 2,500 participants,” in Proceedings of the 2003 Conference on Designing for User Experiences, 2003, pp. 1–15.

S. Wineburg, S. McGrew, “Lateral reading: Reading less and learning more when evaluating digital information,” 2017.

S. S. Birunda, R. K. Devi, “A novel score-based multi- source fake news detection using gradient boosting algorithm,” in 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), 2021, pp. 406– 414, IEEE.

A. Wang, L. M. De Jesus Sanchez, A. Wintner, Y. Zhu, E. Mustafaraj, “Assessing google search’s new features in supporting credibility judgments of unknown websites,” in Proceedings of the 2023 Conference on Human Information Interaction and Retrieval, 2023, pp. 303–307.

H. Keshavarz, M. E. Givi, “Website evaluation frameworks: Is oriented vs. business oriented models,” in 2020 6th International Conference on Web Research (ICWR), 2020, pp. 223–228, IEEE.

Z. E. Serman, J. Sims, “Source credibility theory: Sme hospitality sector blog posting during the covid-19 pandemic,” Information Systems Frontiers, vol. 25, no. 6, pp. 2317–2334, 2023.

S. Padhye, M. Hastak, “A framework to evaluate information and source credibility: International construction decision-making,” Journal of Management in Engineering, vol. 40, no. 1, p. 04023046, 2024.

A. V. Gorodishchev, A. N. Gorodishcheva, D. O. Baigozhina, G. P. Kovalev, “Web 3.0 credibility: Principles for ranking media sources,” in 2023 Communication Strategies in Digital Society Seminar (ComSDS), 2023, pp. 184–188, IEEE.

H. Liu, L. Wang, X. Han, W. Zhang, X. He, “Detecting fake news on social media: A multi-source scoring framework,” in 2020 IEEE 5th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA), 2020, pp. 524–531, IEEE.

M. Fayaz, A. Khan, M. Bilal, S. U. Khan, “Machine learning for fake news classification with optimal feature selection,” Soft Computing, vol. 26, no. 16, pp. 7763–7771, 2022.

I. Ahmad, M. Yousaf, S. Yousaf, M. O. Ahmad, “Fake news detection using machine learning ensemble methods,” Complexity, vol. 2020, pp. 1–11, 2020.

M. Kakol, R. Nielek, A. Wierzbicki, “Understanding and predicting web content credibility using the content credibility corpus,” Information Processing & Management, vol. 53, no. 5, pp. 1043–1061, 2017.

N. Sitaula, C. K. Mohan, J. Grygiel, X. Zhou, R. Zafarani, “Credibilitybased fake news detection,” Disinformation, Misinformation, and Fake News in Social Media: Emerging Research Challenges and Opportunities, pp. 163–182, 2020.

Y. Zhao, L. Zhang, C. Zeng, Y. Chen, W. Lu, N. Song, “Factors influencing online health information credibility: a meta-analysis,” Aslib Journal of Information Management, 2024.

M. M. Bhuiyan, M. Horning, S. W. Lee, T. Mitra, “Nudgecred: Supporting news credibility assessment on social media through nudges,” Proceedings of the ACM on Human-Computer Interaction, vol. 5, no. CSCW2, pp. 1–30, 2021.

M. Bink, S. Zimmerman, D. Elsweiler, “Featured snippets and their influence on users’ credibility judgements,” in Proceedings of the 2022 Conference on Human Information Interaction and Retrieval, 2022, pp. 113–122.

M. Viviani, G. Pasi, “Credibility in social media: opinions, news, and health information—a survey,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 7, no. 5, p. e1209, 2017.

D. Pehlivanoglu, T. Lin, F. Deceus, A. Heemskerk, N. C. Ebner, B. S. Cahill, “The role of analytical reasoning and source credibility on the evaluation of real and fake full-length news articles,” Cognitive Research: Principles and Implications, vol. 6, no. 1, pp. 1–12, 2021.

V. Wagle, K. Kaur, P. Kamat, S. Patil, K. Kotecha, “Explainable ai for multimodal credibility analysis: Case study of online beauty health (mis)- information,” IEEE Access, vol. 9, pp. 127985–128022, 2021.

A. Aker, K. Vincentius, K. Bontcheva, “Credibility and transparency of news sources: Data collection and feature analysis.,” in NewsIR@ SIGIR, 2019, pp. 15–20.

K.-Y. Shin, W. Song, J. Kim, J.-H. Lee, “News credibility scroing: Suggestion of research methodology to determine the reliability of news distributed in sns,” in Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 2019, pp. 737–740.

Y. Lin, S. C. Han, B. H. Kang, “Machine learning for the peer assessment credibility,” in Companion Proceedings of the The Web Conference 2018, 2018, pp. 117–118.

G. Hu, H. Liu, J. Wang, J. Ye, W. Gong, S. Song, “Research on information source detection based on machine learning algorithm,” in 2022 7th International Conference on Intelligent Computing and Signal Processing (ICSP), 2022, pp. 1507–1510, IEEE.

T. H. Chiang, C.-S. Liao, W.-C. Wang, “Investigating the difference of fake news source credibility recognition between ann and bert algorithms in artificial intelligence,” Applied Sciences, vol. 12, no. 15, p. 7725, 2022.

M.-A. Kaufhold, M. Bayer, D. Hartung, C. Reuter, “Design and evaluation of deep learning models for real-time credibility assessment in twitter,” in Artificial Neural Networks and Machine Learning–ICANN 2021: 30th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 14–17, 2021, Proceedings, Part V 30, 2021, pp. 396–408, Springer.

J. A. Nasir, O. S. Khan, I. Varlamis, “Fake news detection: A hybrid cnnrnn based deep learning approach,” International Journal of Information Management Data Insights, vol. 1, no. 1, p. 100007, 2021.

A. Amini, Y. E. Bayiz, A. Ram, R. Marculescu, U. Topcu, “News source credibility assessment: A reddit case study,” arXiv preprint arXiv:2402.10938, 2024.

W. Lifferth, “Fake News. Kaggle,” 2018. [Online]. Available: https://kaggle.com/competitions/fake-news.

K. Shu, D. Mahudeswaran, S. Wang, D. Lee, H. Liu, “Fakenewsnet: A data repository with news content, social context and dynamic information for studying fake news on social media,” arXiv preprint arXiv:1809.01286, 2018.

M. Szpakowski, “FakeNewsCorpus,” 2020. [Online]. Available: http://github.com/several27/FakeNewsCorpus.

M. Zainab, “News credibility dataset,” 2021. [Online]. Available: https://dx.doi.org/10.21227/xjkz-c814, doi: 10.21227/xjkz-c814.