A Survey on Data-Driven Evaluation of Competencies and Capabilities Across Multimedia Environments.

Authors

DOI:

https://doi.org/10.9781/ijimai.2022.10.004

Keywords:

Artificial Intelligence, Competencies, Social, Data Mining, Multimedia
Supporting Agencies
This study was partially funded by the Spanish Government grants IJC2020-044852-I and RYC-2015-18210, co-funded by the European Social Fund, as well as by the COBRA project (10032/20/0035/00), granted by the Spanish Ministry of Defence and by the SCORPION project (21661-PDC-21), granted by the Seneca Foundation of the Region of Murcia, Spain.

Abstract

The rapid evolution of technology directly impacts the skills and jobs needed in the next decade. Users can, intentionally or unintentionally, develop different skills by creating, interacting with, and consuming the content from online environments and portals where informal learning can emerge. These environments generate large amounts of data; therefore, big data can have a significant impact on education. Moreover, the educational landscape has been shifting from a focus on contents to a focus on competencies and capabilities that will prepare our society for an unknown future during the 21st century. Therefore, the main goal of this literature survey is to examine diverse technology-mediated environments that can generate rich data sets through the users’ interaction and where data can be used to explicitly or implicitly perform a data-driven evaluation of different competencies and capabilities. We thoroughly and comprehensively surveyed the state of the art to identify and analyse digital environments, the data they are producing and the capabilities they can measure and/or develop. Our survey revealed four key multimedia environments that include sites for content sharing & consumption, video games, online learning and social networks that fulfilled our goal. Moreover, different methods were used to measure a large array of diverse capabilities such as expertise, language proficiency and soft skills. Our results prove the potential of the data from diverse digital environments to support the development of lifelong and lifewide 21st-century capabilities for the future society.

Downloads

Download data is not yet available.

References

B. S. Bloom, “The new direction in educational research: Alterable variables,” The Journal of Negro Education, vol. 49, no. 3, pp. 337–349, 1980.

C. R. Wolfe, “Creating informal learning environments on the world wide web,” in Learning and teaching on the World Wide Web, Elsevier, 2001, pp. 91–112.

S. Downes, et al., “New technology supporting informal learning,” Journal of emerging technologies in web intelligence, vol. 2, no. 1, pp. 27–33, 2010.

A. Nandi, M. Mandernach, “Hackathons as an informal learning platform,” in Proceedings of the 47th ACM Technical Symposium on Computing Science Education, SIGCSE ’16, New York, NY, USA, 2016, p. 346–351, Association for Computing Machinery.

J. Maldonado-Mahauad, M. Pérez-Sanagustín, R. F. Kizilcec, N. Morales, J. Munoz-Gama, “Mining theory-based patterns from big data: Identifying self-regulated learning strategies in massive open online courses,” Computers in Human Behavior, vol. 80, pp. 179 – 196, 2018, doi: https://doi.org/10.1016/j.chb.2017.11.011.

M. Anshari, Y. Alas, L. S. Guan, “Developing online learning resources: Big data, social networks, and cloud computing to support pervasive knowledge,” Education and Information Technologies, vol. 21, no. 6, pp. 1663–1677, 2016.

R. Kizilcec, C. Brooks, “Diverse Big Data and Randomized Field Experiments in Massive Open Online Courses,” in The Handbook of Learning Analytics, C. Lang, G. Siemens, A. F. Wise, D. Gaševic Eds., Alberta, Canada: Society for Learning Analytics Research (SoLAR), 2017, pp. 211–222, 1 ed.

R. Eynon, “The rise of big data: what does it mean for education, technology, and media research?,” Learning, Media and Technology, vol. 38, no. 3, pp. 237– 240, 2013, doi: 10.1080/17439884.2013.771783.

J.-E. Mai, “Big data privacy: The datafication of personal information,” The Information Society, vol. 32, no. 3, pp. 192–199, 2016.

V. Mayer-Schönberger, K. Cukier, Big data: A revolution that will transform how we live, work, and think. Houghton Mifflin Harcourt, 2013.

J. Van Dijck, “Datafication, dataism and dataveillance: Big data between scientific paradigm and ideology,” Surveillance & Society, vol. 12, no. 2, pp. 197–208, 2014.

H.-U. Otto, H. Ziegler, “Capabilities and education,” Social Work & Society, vol. 4, no. 2, pp. 269–287, 2006.

D. Kember, D. Y. Leung, R. S. Ma, “Characterizing learning environments capable of nurturing generic capabilities in higher education,” Research in Higher Education, vol. 48, no. 5, p. 609, 2007.

M. Pinzone, P. Fantini, S. Perini, S. Garavaglia, M. Taisch, G. Miragliotta, “Jobs and skills in industry 4.0: An exploratory research,” in Advances in Production Management Systems. The Path to Intelligent, Collaborative and Sustainable Manufacturing, Cham, 2017, pp. 282–288, Springer International Publishing.

A. Smith, J. Anderson, “AI, robotics, and the future of jobs,” Pew Research Center, vol. 6, p. 51, 2014.

N. P. Stromquist, Education in a globalized world: The connectivity of economic power, technology, and knowledge. Rowman & Littlefield, 2002.

C. Redecker, M. Leis, M. Leendertse, Y. Punie, G. Gijsbers, P. Kirschner, S. Stoyanov, B. Hoogveld, “The future of learning: New ways to learn new skills for future jobs,” Results from an online expert consultation. Technical Note JRC60869, JRC-IPTS, Seville, 2010.

J. C.-Y. Sun, R. Rueda, “Situational interest, computer self-efficacy and self-regulation: Their impact on student engagement in distance education,” British Journal of educational technology, vol. 43, no. 2, pp. 191–204, 2012.

R. Nagarajan, R. Prabhu, “Competence and capability: A new look,” International Journal of Management, vol. 6, no. 6, pp. 7–11, 2015.

R. Hipkins, “Competencies or capabilities,” He Whakaaro An, Se2, vol. 3, pp. 55–57, 2013.

R. E. Boyatzis, The competent manager: A model for effective performance. John Wiley & Sons, 1982.

R. Henderson, I. Cockburn, “Measuring competence? exploring firm effects in pharmaceutical research,” Strategic management journal, vol. 15, no. S1, pp. 63–84, 1994.

J. F. Lozano, A. Boni, J. Peris, A. Hueso, “Competencies in higher education: A critical analysis from the capabilities approach,” Journal of Philosophy of Education, vol. 46, no. 1, pp. 132–147, 2012.

V. Vathanophas, “Competency requirements for effective job performance in Thai public sector,” Contemporary management research, vol. 3, no. 1, p. 45, 2007.

J. Macnamara, “Competence, competencies and/or capabilities for public communication? a public sector study,” Asia Pacific Public Relations Journal, vol. 19, pp. 16–40, 2018.

P. Morgan, “The concept of capacity,” European Centre for Development Policy Management, pp. 1–19, 2006.

L. Vincent, “Differentiating competence, capability and capacity,” Innovating Perspectives, vol. 16, no. 3, pp. 1–2, 2008.

M. Z. Al-Taie, S. Kadry, A. I. Obasa, “Understanding expert finding systems: domains and techniques,” Social Network Analysis and Mining, vol. 8, no. 1, p. 57, 2018.

O. Husain, N. Salim, R. A. Alias, S. Abdelsalam, A. Hassan, “Expert finding systems: A systematic review,” Applied Sciences, vol. 9, no. 20, p. 4250, 2019.

I. Srba, M. Bielikova, “A comprehensive survey and classification of approaches for community question answering,” ACM Transactions on the Web (TWEB), vol. 10, no. 3, pp. 1–63, 2016.

T. M. Connolly, E. A. Boyle, E. MacArthur, T. Hainey, J. M. Boyle, “A systematic literature review of empirical evidence on computer games and serious games,” Computers & Education, vol. 59, no. 2, pp. 661–686, 2012, doi: https://doi.org/10.1016/j.compedu.2012.03.004

E. A. Boyle, T. Hainey, T. M. Connolly, G. Gray, J. Earp, M. Ott, T. Lim, M. Ninaus, C. Ribeiro, J. Pereira, “An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games,” Computers & Education, vol. 94, pp. 178 – 192, 2016, doi: https://doi.org/10.1016/j.compedu.2015.11.003

T. Hainey, T. M. Connolly, E. A. Boyle, A. Wilson, A. Razak, “A systematic literature review of games- based learning empirical evidence in primary education,” Computers & Education, vol. 102, pp. 202–223, 2016.

X. Wei, N. Saab, W. Admiraal, “Assessment of cognitive, behavioral, and affective learning outcomes in massive open online courses: A systematic literature review,” Computers & Education, vol. 163, p. 104097, 2021, doi: https://doi.org/10.1016/j.compedu.2020.104097

M. J. Page, D. Moher, P. M. Bossuyt, I. Boutron, T. C. Hoffmann, C. D. Mulrow, L. Shamseer, J. M. Tetzlaff, E. A. Akl, S. E. Brennan, et al., “PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews,” BMJ, vol. 372, 2021.

R. Huelin, I. Iheanacho, K. Payne, K. Sandman, “What’s in a name? systematic and non-systematic literature reviews, and why the distinction matters,” The evidence, pp. 34–37, 2015.

C. Wohlin, “Guidelines for snowballing in systematic literature studies and a replication in software engineering,” in Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, EASE ’14, New York, NY, USA, 2014, Association for Computing Machinery.

J. Randolph, “A guide to writing the dissertation literature review,” Practical Assessment, Research, and Evaluation, vol. 14, no. 1, p. 13, 2009.

F. Riahi, Z. Zolaktaf, M. Shafiei, E. Milios, “Finding expert users in community question answering,” in Proceedings of the 21st International Conference on World Wide Web, WWW ’12 Companion, New York, NY, USA, 2012, p. 791–798, Association for Computing Machinery.

D. Movshovitz-Attias, Y. Movshovitz-Attias, P. Steenkiste, C. Faloutsos, “Analysis of the reputation system and user contributions on a question answering website: StackOverflow,” in 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013), 2013, pp. 886–893, IEEE.

Y. Xu, D. Zhou, S. Lawless, “Inferring your expertise from Twitter: Integrating sentiment and topic relatedness,” in 2016 IEEE/WIC/ACM International Conference on Web Intelligence (WI), 2016, pp. 121–128, IEEE.

L. Akritidis, D. Katsaros, P. Bozanis, “Identifying the productive and influential bloggers in a community,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 41, no. 5, pp. 759–764, 2011.

Y. Bae, H. Lee, “Sentiment analysis of Twitter audiences: Measuring the positive or negative influence of popular twitterers,” Journal of the American Society for Information Science and Technology, vol. 63, no. 12, pp. 2521–2535, 2012.

C. Bigonha, T. N. Cardoso, M. M. Moro, M. A. Gonçalves, V. A. Almeida, “Sentiment-based influence detection on Twitter,” Journal of the Brazilian Computer Society, vol. 18, no. 3, pp. 169–183, 2012.

M. Cha, H. Haddadi, F. Benevenuto, P. K. Gummadi, et al., “Measuring user influence in Twitter: The million follower fallacy,” Icwsm, vol. 10, no. 10-17, p. 30, 2010.

H. U. Khan, A. Daud, “Finding the top influential bloggers based on productivity and popularity features,” New Review of Hypermedia and Multimedia, vol. 23, no. 3, pp. 189–206, 2017.

S. Nagrecha, J. Z. Dillon, N. V. Chawla, “MOOC dropout prediction: Lessons learned from making pipelines interpretable,” in Proceedings of the 26th International Conference on World Wide Web Companion, WWW ’17 Companion, Republic and Canton of Geneva, CHE, 2017, p. 351–359, International World Wide Web Conferences Steering Committee.

N. Dmoshinskaia, “Dropout prediction in MOOCs: using sentiment analysis of users’ comments to predict engagement.,” Master’s thesis, University of Twente, 2016.

S. Patil, K. Lee, “Detecting experts on Quora: by their activity, quality of answers, linguistic characteristics and temporal behaviors,” Social network analysis and mining, vol. 6, no. 1, p. 5, 2016.

D. Choi, J. Han, T. Chung, Y.-Y. Ahn, B.-G. Chun, T. T. Kwon, “Characterizing conversation patterns in Reddit: From the perspectives of content properties and user participation behaviors,” in Proceedings of the 2015 ACM on Conference on Online Social Networks, COSN ’15, New York, NY, USA, 2015, p. 233–243, Association for Computing Machinery.

J. Yang, K. Tao, A. Bozzon, G.-J. Houben, “Sparrows and owls: Characterisation of expert behaviour in StackOverflow,” in User Modeling, Adaptation, and Personalization, Cham, 2014, pp. 266–277, Springer, Springer International Publishing.

E. Malherbe, M.-A. Aufaure, “Bridge the terminology gap between recruiters and candidates: A multilingual skills base built from social media and linked data,” in 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2016, pp. 583–590, IEEE.

G. Zhou, S. Lai, K. Liu, J. Zhao, “Topic-sensitive probabilistic model for expert finding in question answer communities,” in Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM ’12, New York, NY, USA, 2012, p. 1662–1666, Association for Computing Machinery.

P. Jurczyk, E. Agichtein, “Discovering authorities in question answer communities by using link analysis,” in Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, 2007, pp. 919–922.

N. Raj, L. Dey, B. Gaonkar, “Expertise prediction for social network platforms to encourage knowledge sharing,” in Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01, WI-IAT ’11, USA, 2011, p. 380–383, IEEE Computer Society.

A. Raikos, P. Waidyasekara, “How useful is YouTube in learning heart anatomy?,” Anatomical sciences education, vol. 7, no. 1, pp. 12–18, 2014.

A. Pal, A. Herdagdelen, S. Chatterji, S. Taank, Chakrabarti, “Discovery of topical authorities in Instagram,” in Proceedings of the 25th International Conference on World Wide Web, WWW ’16, Republic and Canton of Geneva, CHE, 2016, p. 1203–1213, International World Wide Web Conferences Steering Committee.

J. T. Hertel, N. M. Wessman-Enzinger, “Examining Pinterest as a curriculum resource for negative integers: An initial investigation,” Education Sciences, vol. 7, no. 2, p. 45, 2017.

A.-M. Popescu, K. Y. Kamath, J. Caverlee, “Mining potential domain expertise in Pinterest,” in UMAP workshops, 2013.

J. Oliveira, M. Viggiato, E. Figueiredo, “How well do you know this library? mining experts from source code analysis,” in Proceedings of the XVIII Brazilian Symposium on Software Quality, SBQS’19, New York, NY, USA, 2019, p. 49–58, Association for Computing Machinery.

R. Saxena, N. Pedanekar, “I know what you coded last summer: Mining candidate expertise from GitHub repositories,” in Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW ’17 Companion, New York, NY, USA, 2017, p. 299–302, Association for Computing Machinery.

P. A. Martínez, M. J. Gómez, J. A. Ruipérez-Valiente, G. Martínez Pérez, Y. J. Kim, “Visualizing educational game data: A case study of visualizations to support teachers,” in Learning Analytics Summer Institute Spain 2020: Learning Analytics. Time for Adoption?, Jun 2020, pp. 97–111, CEUR Workshop Proceedings.

E. Harpstead, T. Zimmermann, N. Nagapan, J. J. Guajardo, R. Cooper, T. Solberg, D. Greenawalt, “What drives people: Creating engagement profiles of players from game log data,” in Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play, CHI PLAY ’15, New York, NY, USA, 2015, p. 369–379, Association for Computing Machinery.

A. J. Lesser, Video game technology and learning in the music classroom. PhD dissertation, Teachers College, Columbia University, 2019.

P. Kantharaju, K. Alderfer, J. Zhu, B. Char, B. Smith, S. Ontañón, “Tracing player knowledge in a parallel programming educational game,” 2019.

F. Chen, Y. Cui, M.-W. Chu, “Utilizing game analytics to inform and validate digital game-based assessment with evidence-centered game design: A case study,” International Journal of Artificial Intelligence in Education, vol. 30, no. 3, pp. 481–503, 2020.

J. A. Ruipérez-Valiente, M. Gaydos, L. Rosenheck, Y. J. Kim, E. Klopfer, “Patterns of engagement in an educational massive multiplayer online game: A multidimensional view,” IEEE Transactions on Learning Technologies, 2020.

Y. J. Kim, J. A. Ruipérez-Valiente, “Data-driven game design: The case of difficulty in educational games,” in European Conference on Technology Enhanced Learning, 2020, pp. 449–454, Springer.

M.-T. Cheng, Y.-W. Lin, H.-C. She, “Learning through playing Virtual Age: Exploring the interactions among student concept learning, gaming performance, in-game behaviors, and the use of in-game characters,” Computers & Education, vol. 86, pp. 18–29, 2015.

J. Kang, M. Liu, W. Qu, “Using gameplay data to examine learning behavior patterns in a serious game,” Computers in Human Behavior, vol. 72, pp. 757–770, 2017.

W. Westera, R. Nadolski, H. Hummel, “Serious gaming analytics: What students’ log files tell us about gaming and learning,” International Journal of Serious Games, vol. 1, Jun. 2014, doi: 10.17083/ijsg.v1i2.9.

T. Daradoumis, R. Bassi, F. Xhafa, S. Caballé, “A review on massive e-learning (MOOC) design, delivery and assessment,” in 2013 Eighth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing, 2013, pp. 208–213.

S. Crossley, L. Paquette, M. Dascalu, D. S. McNamara, R. S. Baker, “Combining click-stream data with NLP tools to better understand MOOC completion,” in Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, LAK ’16, New York, NY, USA, 2016, p. 6–14, Association for Computing Machinery.

R. Zhao, V. Li, H. Barbosa, G. Ghoshal, M. E. Hoque, “Semi-automated 8 collaborative online training module for improving communication skills,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 1, no. 2, pp. 1–20, 2017.

R. Reddick, “Using a Glicko-based algorithm to measure in-course learning,” International Educational Data Mining Society, 2019.

X. Wang, D. Yang, M. Wen, K. Koedinger, C. P. Rosé, “Investigating how student's cognitive behavior in MOOC discussion forums affect learning gains.,” International Educational Data Mining Society, 2015.

J. Kim, P. J. Guo, D. T. Seaton, P. Mitros, K. Z. Gajos, R. C. Miller, “Understanding in-video dropouts and interaction peaks in online lecture videos,” in Proceedings of the First ACM Conference on Learning @ Scale Conference, L@S ’14, New York, NY, USA, 2014, p. 31–40, Association for Computing Machinery.

D. Huynh, L. Zuo, H. Iida, “Analyzing gamification of “Duolingo” with focus on its course structure,” in International Conference on Games and Learning Alliance, 2016, pp. 268–277, Springer.

S. Chootongchai, N. Songkram, “Design and development of seci and moodle online learning systems to enhance thinking and innovation skills for higher education learners,” International Journal of Emerging Technologies in Learning (iJET), vol. 13, no. 03, pp. 154–172, 2018.

B. Hightower, C. Rawl, M. Schutt, “Collaborations for delivering the library to students through WebCT,” Reference Services Review, 2007.

H. Tanaka, S. Sakti, G. Neubig, T. Toda, H. Negoro, H. Iwasaka, S. Nakamura, “Automated social skills trainer,” in Proceedings of the 20th International Conference on Intelligent User Interfaces, IUI ’15, New York, NY, USA, 2015, p. 17–27, Association for Computing Machinery.

P. Pham, J. Wang, “Attentivelearner: Improving mobile MOOC learning via implicit heart rate tracking,” in Artificial Intelligence in Education, Cham, 2015, pp. 367–376, Springer, Springer International Publishing.

C. Yin, F. Okubo, A. Shimada, M. Oi, S. Hirokawa, H. Ogata, “Identifying and analyzing the learning behaviors of students using e-books,” in Doctoral Student Consortium (DSC) - Proceedings of the 23rd International Conference on Computers in Education, ICCE 2015, 2015, pp. 118–120, Asia-Pacific Society for Computers in Education.

A. Shimada, K. Mouri, H. Ogata, “Real-time learning analytics of e-book operation logs for on-site lecture support,” in 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT), 2017, pp. 274–275, IEEE.

D. M. Boyd, N. B. Ellison, “Social network sites: Definition, history, and scholarship,” Journal of computer-mediated communication, vol. 13, no. 1, pp. 210–230, 2007.

J. Pastor-Galindo, M. Zago, P. Nespoli, S. López Bernal, A. Huertas, M. Pérez, J. A. Ruipérez-Valiente, G. Martínez Pérez, F. Gómez Mármol, “Spotting political social bots in Twitter: A use case of the 2019 spanish general election,” IEEE Transactions on Network and Service Management, vol. 17, pp. 2156–2170, 10 2020, doi: 10.1109/TNSM.2020.3031573.

E. Özdemir, “Promoting EFL learners” intercultural communication effectiveness: a focus on Facebook,” Computer Assisted Language Learning, vol. 30, no. 6, pp. 510–528, 2017.

W. Orawiwatnakul, S. Wichadee, “Achieving better learning performance through the discussion activity in Facebook,” Turkish Online Journal of Educational Technology-TOJET, vol. 15, no. 3, pp. 1–8, 2016.

X. Yan, J. Yang, M. Obukhov, L. Zhu, J. Bai, S. Wu, Q. He, “Social skill validation at LinkedIn,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’19, New York, NY, USA, 2019, p. 2943–2951, Association for Computing Machinery.

B. Muros-Ruiz, Y. Aragón-Carretero, A. Bustos- Jiménez, “Youth's usage of leisure time with video games and social networks,” Comunicar: Revista Científica de Comunicación y Educación, vol. 20, no. 40, pp. 31–39, 2013.

M. Boukes, “Social network sites and acquiring current affairs knowledge: The impact of Twitter and Facebook usage on learning about the news,” Journal of Information Technology & Politics, vol. 16, no. 1, pp. 36–51, 2019.

T. P. Alloway, R. G. Alloway, “The impact of engagement with social networking sites (SNSs) on cognitive skills,” Computers in Human Behavior, vol. 28, no. 5, pp. 1748–1754, 2012.

M. Niemelä, T. Kärkkäinen, S. Äyrämö, M. Ronimus, U. Richardson, H. Lyytinen, “Game learning analytics for understanding reading skills in transparent writing system,” British Journal of Educational Technology, 02 2020, doi: 10.1111/bjet.12916.

A. I. Wang, A. Lieberoth, “The effect of points and audio on concentration, engagement, enjoyment, learning, motivation, and classroom dynamics using Kahoot!,” in European Conference on Games Based Learning, vol. 20, 2016, Academic Conferences International Limited.

M. Bouguessa, L. B. Romdhane, “Identifying authorities in online communities,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 6, no. 3, pp. 1–23, 2015.

G. Guerrero, E. Sarchi, F. Tapia, “Predict the personality of Facebook profiles using automatic learning techniques and BFI test,” in New Knowledge in Information Systems and Technologies, Cham, 2019, pp. 482–493, Springer International Publishing.

R. Nielek, O. Jarczyk, K. Pawlak, L. Bukowski, R. Bartusiak, A. Wierzbicki, “Choose a job you love: predicting choices of GitHub developers,” in 2016 IEEE/WIC/ACM International Conference on Web Intelligence (WI), 2016, pp. 200–207, IEEE.

A. Cohen, U. Shimony, R. Nachmias, T. Soffer, “Active learners’ characterization in MOOC forums and their generated knowledge,” British Journal of Educational Technology, vol. 50, no. 1, pp. 177–198, 2019.

J. Han, D. Choi, A.-Y. Choi, J. Choi, T. Chung, T. T. Kwon, J.-Y. Rha, C.-N. Chuah, “Sharing topics in Pinterest: Understanding content creation and diffusion behaviors,” in Proceedings of the 2015 ACM on Conference on Online Social Networks, COSN ’15, New York, NY, USA, 2015, p. 245–255, Association for Computing Machinery.

F. Calefato, F. Lanubile, B. Vasilescu, “A large-scale, in-depth analysis of developers’ personalities in the apache ecosystem,” Information and Software Technology, vol. 114, pp. 1–20, 2019.

J. Baumgartner, S. Zannettou, B. Keegan, M. Squire, J. Blackburn, “The Pushshift Reddit dataset,” in Proceedings of the International AAAI Conference on Web and Social Media, vol. 14, 2020, pp. 830–839.

B. Vasilescu, V. Filkov, A. Serebrenik, “StackOverflow and GitHub: Associations between software development and crowdsourced knowledge,” in 2013 International Conference on Social Computing, 2013, pp. 188–195, IEEE.

W. H. Lim, M. J. Carman, S.-M. J. Wong, “Estimating relative user expertise for content quality prediction on Reddit,” in Proceedings of the 28th ACM Conference on Hypertext and Social Media, HT ’17, New York, NY, USA, 2017, p. 55–64, Association for Computing Machinery.

C. Mhamdi, M. Al-Emran, S. A. Salloum, Text Mining and Analytics: A Case Study from News Channels Posts on Facebook, pp. 399–415. Cham: Springer International Publishing, 2018.

W. Xing, F. Gao, “Exploring the relationship between online discourse and commitment in Twitter professional learning communities,” Computers & Education, vol. 126, pp. 388–398, 2018.

T. Hecking, I.-A. Chounta, H. U. Hoppe, “Investigating social and semantic user roles in MOOC discussion forums,” in Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, LAK ’16, New York, NY, USA, 2016, p. 198–207, Association for Computing Machinery.

A. Pal, S. Counts, “Identifying topical authorities in microblogs,” in Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, WSDM ’11, New York, NY, USA, 2011, p. 45–54, Association for Computing Machinery.

Z.-J. Yang, J. Lin, Y.-S. Yang, “Identification of network behavioral characteristics of high-expertise users in interactive innovation: The case of forum autohome,” Asia Pacific Management Review, 2020, doi: https://doi.org/10.1016/j.apmrv.2020.06.002

H. Zhu, H. Cao, H. Xiong, E. Chen, J. Tian, “Towards expert finding by leveraging relevant categories in authority ranking,” in Proceedings of the 20th ACM international conference on Information and knowledge management, 2011, pp. 2221–2224.

C. MACLEOD, “Evaluating student use of Duolingo, an online self—study platform,” JOURNAL OF ATOMI UNIVERSITY FACULTY OF LITERATURE, no. 54, pp. A49–A67, 2019.

S. Loewen, D. Crowther, D. R. Isbell, K. M. Kim, J. Maloney, Z. F. Miller, H. Rawal, “Mobile-assisted language learning: A Duolingo case study,” ReCALL, vol. 31, no. 3, pp. 293–311, 2019.

M. Liu, J. Lee, J. Kang, S. Liu, “What we can learn from the data: A multiple-case study examining behavior patterns by students with different characteristics in using a serious game,” Technology, Knowledge and Learning, vol. 21, no. 1, pp. 33–57, 2016.

Y. Sun, C. Xin, “Using Coursera clickstream data to improve online education for software engineering,” in Proceedings of the ACM Turing 50th Celebration Conference - China, ACM TUR-C ’17, New York, NY, USA, 2017, Association for Computing Machinery.

L. Breslow, D. E. Pritchard, J. DeBoer, G. S. Stump, A. D. Ho, D. T. Seaton, “Studying learning in the worldwide classroom research into edX’s first MOOC,” Research & Practice in Assessment, vol. 8, pp. 13–25, 2013.

G. Alexandron, J. A. Ruipérez-Valiente, Z. Chen, P. J. Muñoz-Merino, D. E. Pritchard, “Copying@Scale: Using harvesting accounts for collecting correct answers in a MOOC,” Computers & Education, vol. 108, pp. 96– 114, 2017.

I. Naim, M. I. Tanveer, D. Gildea, M. E. Hoque, “Automated prediction and analysis of job interview performance: The role of what you say and how you say it,” in 2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG), vol. 1, 2015, pp. 1–6, IEEE.

S. Peng., K. Nagao., “Automatic evaluation of presenters’ discussion performance based on their heart rate,” in Proceedings of the 10th International Conference on Computer Supported Education - Volume 1: CSEDU, 2018, pp. 27–34, INSTICC, SciTePress.

A. Shimada, Y. Taniguchi, F. Okubo, S. Konomi, H. Ogata, “Online change detection for monitoring individual student behavior via clickstream data on e-book system,” in Proceedings of the 8th International Conference on Learning Analytics and Knowledge, LAK ’18, New York, NY, USA, 2018, p. 446–450, Association for Computing Machinery.

F. J. Gutierrez, J. Simmonds, N. Hitschfeld, C. Casanova, C. Sotomayor, V. Peña Araya, “Assessing software development skills among k-6 learners in a project-based workshop with scratch,” in Proceedings of the 40th International Conference on Software Engineering: Software Engineering Education and Training, ICSE-SEET ’18, New York, NY, USA, 2018, p. 98–107, Association for Computing Machinery.

C. G. Brinton, M. Chiang, “MOOC performance prediction via clickstream data and social learning networks,” in 2015 IEEE Conference on Computer Communications (INFOCOM), 2015, pp. 2299–2307, IEEE.

M. Bouguessa, B. Dumoulin, S. Wang, “Identifying authoritative actors in question-answering forums: The case of Yahoo! Answers,” in Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’08, New York, NY, USA, 2008, p. 866– 874, Association for Computing Machinery.

W.-C. Kao, D.-R. Liu, S.-W. Wang, “Expert finding in question-answering websites: A novel hybrid approach,” in Proceedings of the 2010 ACM symposium on applied computing, SAC ’10, New York, NY, USA, 2010, p. 867–871, Association for Computing Machinery.

A. Borodin, G. O. Roberts, J. S. Rosenthal, P. Tsaparas, “Link analysis ranking: algorithms, theory, and experiments,” ACM Transactions on Internet Technology (TOIT), vol. 5, no. 1, pp. 231–297, 2005.

R. W. Herling, “Operational definitions of expertise and competence,” Advances in Developing Human Resources, vol. 2, no. 1, pp. 8–21, 2000, doi: 10.1177/152342230000200103.

A. Santos, M. Souza, J. Oliveira, E. Figueiredo, “Mining software repositories to identify library experts,” in Proceedings of the VII Brazilian Symposium on Software Components, Architectures, and Reuse, SBCARS ’18, New York, NY, USA, 2018, p. 83–91, Association for Computing Machinery.

J. A. E. Montandon, L. L. Silva, M. T. Valente, “Identifying experts in software libraries and frameworks among GitHub users,” in Proceedings of the 16th International Conference on Mining Software Repositories, MSR ’19, 2019, p. 276–287, IEEE, IEEE Press.

C. Hauff, G. Gousios, “Matching GitHub developer profiles to job advertisements,” in 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories, 2015, pp. 362–366.

M. S. Faisal, A. Daud, A. U. Akram, R. A. Abbasi, N. R. Aljohani, I. Mehmood, “Expert ranking techniques for online rated forums,” Computers in Human Behavior, vol. 100, pp. 168–176, 2019, doi: https://doi.org/10.1016/j.chb.2018.06.013.

D. van Dijk, M. Tsagkias, M. de Rijke, “Early detection of topical expertise in community question answering,” in Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’15, New York, NY, USA, 2015, p. 995–998, Association for Computing Machinery.

E. Constantinou, G. M. Kapitsaki, “Identifying developers’ expertise in social coding platforms,” in 2016 42nd Euromicro Conference on Software Engineering and Advanced Applications (SEAA), 2016, pp. 63–67.

M. A. Vogel, Leveraging information technology competencies and capabilities for a competitive advantage. PhD dissertation, 2005.

D.-R. Liu, Y.-H. Chen, W.-C. Kao, H.-W. Wang, “Integrating expert profile, reputation and link analysis for expert finding in question-answering websites,” Information processing & management, vol. 49, no. 1, pp. 312–329, 2013.

Z. Zhao, L. Zhang, X. He, W. Ng, “Expert finding for question answering via graph regularized matrix completion,” IEEE Transactions on Knowledge and Data Engineering, vol. 27, no. 4, pp. 993–1004, 2014.

A. Omidvar, M. Garakani, H. R. Safarpour, “Context based user ranking in forums for expert finding using wordnet dictionary and social network analysis,” Information Technology and Management, vol. 15, no. 1, pp. 51–63, 2014.

A. Pal, F. M. Harper, J. A. Konstan, “Exploring question selection bias to identify experts and potential experts in community question answering,” ACM Transactions on Information Systems (TOIS), vol. 30, no. 2, pp. 1–28, 2012.

V. V. Vydiswaran, M. Reddy, “Identifying peer experts in online health forums,” BMC medical informatics and decision making, vol. 19, no. 3, p. 68, 2019.

H. Zhu, E. Chen, H. Xiong, H. Cao, J. Tian, “Ranking user authority with relevant knowledge categories for expert finding,” World Wide Web, vol. 17, no. 5, pp. 1081–1107, 2014.

A. Abdaoui, J. Azé, S. Bringay, N. Grabar, P. Poncelet, “Expertise in french health forums,” Health informatics journal, vol. 25, no. 1, pp. 17–26, 2019.

C. G. Brinton, S. Buccapatnam, M. Chiang, H. V. Poor, “Mining MOOC clickstreams: On the relationship between learner behavior and performance,” 2015.

A. Bozkurt, B. Aydin, A. Taşkıran, Koral Gümüşoğlu, “Improving creative writing skills of EFL learners through microblogging,” The Online Journal of New Horizons in Education, vol. 6, pp. 88–98, 06 2016.

T. Gonulal, “The use of Instagram as a mobile-assisted language learning tool,” Contemporary Educational Psychology, vol. 10, pp. 309–323, 07 2019, doi: 10.30935/cet.590108.

R. Vesselinov, J. Grego, “Duolingo effectiveness study,” City University of New York, USA, vol. 28, 2012.

Z. Liu, G. Xu, T. Liu, W. Fu, Y. Qi, W. Ding, Y. Song, C. Guo, C. Kong, S. Yang, et al., “Dolphin: A spoken language proficiency assessment system for elementary education,” Apr 2020, pp. 2641–2647, ACM.

J. Dixon, C. Belnap, C. Albrecht, K. Lee, “The importance of soft skills,” Corporate Finance Review, vol. 14, pp. 35–38, May 2010.

B. Cimatti, “Definition, development, assessment of soft skills and their role for the quality of organizations and enterprises.,” International Journal for quality research, vol. 10, no. 1, 2016.

T. Strobach, P. A. Frensch, T. Schubert, “Video game practice optimizes executive control skills in dual- task and task switching situations,” Acta Psychologica, vol. 140, no. 1, pp. 13–24, 2012.

H.-S. Hsiao, C.-S. Chang, C.-Y. Lin, P.-M. Hu, “Development of children’s creativity and manual skills within digital game-based learning environment,” Journal of Computer Assisted Learning, vol. 30, no. 4, pp. 377–395, 2014.

D. R. Woods, A. N. Hrymak, R. R. Marshall, P. E. Wood, C. M. Crowe, T. W. Hoffman, J. D. Wright, P. A. Taylor, K. A. Woodhouse, C. K. Bouchard, “Developing problem solving skills: The McMaster problem solving program,” Journal of Engineering Education, vol. 86, no. 2, pp. 75–91, 1997.

H.-C. Chu, C.-M. Hung, “Effects of the digital game-development approach on elementary school students’ learning motivation, problem solving, and learning achievement,” International Journal of Distance Education Technologies (IJDET), vol. 13, no. 1, pp. 87–102, 2015.

M. Scriven, R. Paul, “Defining critical thinking. the critical thinking community: foundation for critical thinking,” 2007.

A. Pal, R. Farzan, J. A. Konstan, R. E. Kraut, “Early detection of potential experts in question answering communities,” in International Conference on User Modeling, Adaptation, and Personalization, 2011, pp. 231–242, Springer.

C. Unger, D. Murthy, A. Acker, I. Arora, A. Chang, “Examining the evolution of mobile social payments in Venmo,” in International Conference on Social Media and Society, SMSociety’20, New York, NY, USA, 2020, p. 101–110, Association for Computing Machinery.

M. Yuan, L. Zhang, X.-Y. Li, H. Xiong, “Comprehensive and efficient data labeling via adaptive model scheduling,” in 2020 IEEE 36th International Conference on Data Engineering (ICDE), 2020, pp. 1858–1861, IEEE.

D. Papp, G. Szűcs, Z. Knoll, “Machine preparation for human labelling of hierarchical train sets by spectral clustering,” in 2019 10th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), 2019, pp. 157–162, IEEE.

J. Kang, S. Liu, M. Liu, Tracking Students’ Activities in Serious Games, pp. 125–137. Cham: Springer International Publishing, 2017.

M. Oczak, K. Maschat, D. Berckmans, E. Vranken, J. Baumgartner, “Can an automated labelling method based on accelerometer data replace a human labeller?–postural profile of farrowing sows,” Computers and Electronics in Agriculture, vol. 127, pp. 168–175, 2016.

C. A. Gomez-Uribe, N. Hunt, “The Netflix recommender system: Algorithms, business value, and innovation,” ACM Transactions on Management Information Systems (TMIS), vol. 6, no. 4, pp. 1–19, 2015.

A. Emerson, N. Henderson, J. Rowe, W. Min, S. Lee, J. Minogue, J. Lester, “Investigating visitor engagement in interactive science museum exhibits with multimodal Bayesian hierarchical models,” in Artificial Intelligence in Education, Cham, 2020, pp. 165– 176, Springer International Publishing.

A. K. Cooper, S. Coetzee, “On the ethics of using publicly-available data,” in Responsible Design, Implementation and Use of Information and Communication Technology, Cham, 2020, pp. 159–171, Springer International Publishing.

C. Kuhlman, L. Jackson, R. Chunara, “No computation without representation: Avoiding data and algorithm biases through diversity,” arXiv preprint arXiv:2002.11836, 2020.

M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, S. Venkatasubramanian, “Certifying and removing disparate impact,” in Proceedings of the 21st ACM SIGKDD international conference on knowledge discovery and data mining, 2015, pp. 259–268.

Z. Obermeyer, B. Powers, C. Vogeli, S. Mullainathan, “Dissecting racial bias in an algorithm used to manage the health of populations,” Science, vol. 366, no. 6464, pp. 447–453, 2019.

Downloads

Published

2023-12-01
Metrics
Views/Downloads
  • Abstract
    171
  • PDF
    14

How to Cite

Strukova, S., Ruipérez Valiente, J. A., and Gómez Mármol, F. (2023). A Survey on Data-Driven Evaluation of Competencies and Capabilities Across Multimedia Environments. International Journal of Interactive Multimedia and Artificial Intelligence, 8(4), 182–201. https://doi.org/10.9781/ijimai.2022.10.004