A Benchmark for the UEQ+ Framework: Construction of a Simple Tool to Quickly Interpret UEQ+ KPIs.

Anna Lena Meiners; Martin Schrepp; Andreas Hinderks; Jörg Thomaschewski

doi:10.9781/ijimai.2023.05.003

Authors

Anna Lena Meiners University of Applied Sciences Emden/Leer.
Martin Schrepp SAP SE.
Andreas Hinderks University of Sevilla.
Jörg Thomaschewski University of Applied Sciences Emden/Leer.

DOI:

https://doi.org/10.9781/ijimai.2023.05.003

Keywords:

Benchmark, UEQ, User Experience, UX

Abstract

Questionnaires are a highly efficient method to compare the user experience (UX) of different interactive products or versions of a single product. Concretely, they allow us to evaluate the UX easily and to compare different products with a numeric UX score. However, often only one UX score from a single evaluated product is available. Without a comparison to other measurements, it is difficult to interpret an individual score, e.g. to decide whether a product’s UX is good enough to compete in the market. Many questionnaires offer benchmarks to support researchers in these cases. A benchmark is the result of a larger set of product evaluations performed with the same questionnaire. The score obtained from a single product evaluation can be compared to the scores from this benchmark data set to quickly interpret the results. In this paper, the first benchmark for the UEQ+ (User Experience Questionnaire +) is presented, which was created using 3.290 UEQ+ responses for 26 successful software products. The UEQ+ is a modular framework that contains a high number of validated user experience scales that can be combined to form a UX questionnaire. Currently, no benchmark is available for this framework, making the benchmark constructed in this paper a valuable interpretation tool for UEQ+ questionnaires.

Downloads

Download data is not yet available.

References

M. Thüring, and S. Mahlke, ”Usability, aesthetics and emotions in human–technology interaction,” International journal of psychology, vol. 42, no. 4, 2007, pp. 253-264.

J. R. Lewis, and J. Sauro, “Usability and user experience: Design and evaluation,” Handbook of Human Factors and Ergonomics, 2021, pp. 972-1015.

M. Schrepp, User Experience Questionnaires: How to use questionnaires to measure the user experience of your products?, KDP, 2021, ISBN-13: 979-8736459766.

J. Preece, Y. Rogers, and H. Sharpe, “Interaction design: Beyond human-computer interaction,” Wiley, New York, 2002.

M. Hassenzahl, “The effect of perceived hedonic quality on product appealingness,” International Journal of Human-Computer Interaction, vol. 13, no. 4, 2001, pp. 481-499.

A.-L. Meiners, J. Kollmorgen, M. Schrepp, and J. Thomaschewski, “Which UX aspects are important for a software product? Importance ratings of UX aspects for software products for measurement with the UEQ+,” in Proceedings of Mensch und Computer 2021 (MuC ‘21), Association for Computing Machinery, New York, NY, USA, 2021, pp. 136–139.

H. B. Santoso and M. Schrepp, “The impact of culture and product on the subjective importance of user experience aspects,” Heliyon, vol. 5, no. 9, 2019, doi: 10.1016/j.heliyon.2019.e02434.

M. Schrepp, J. Kollmorgen, A.-L. Meiners, A. Hinderks, D. Winter, H. B. Santoso, J. Thomaschewski. On the Importance of UX Quality Aspects for Different Product Categories, International Journal of Interactive Multimedia and Artificial Intelligence, (2023), http://dx.doi.org/10.9781/ijimai.2023.03.001.

M. Schrepp, and J. Thomaschewski, “Design and Validation of a Framework for the Creation of User Experience Questionnaires,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 5, no. 7, 2019, doi: 10.9781/ijimai.2019.06.006.

H. Sandkühler, M. Schrepp, and J. Thomaschewski, “UX Messung mithilfe des UEQ+ Frameworks (Measuring UX with the UEQ+ framework),”, in Mensch und Computer 2020 - Workshopband, Gesellschaft für Informatik, Bonn, 2020.

M. Schrepp, H. Sandkühler, and J. Thomaschewski, “How to create short forms of UEQ+ based questionnaires?,” in Mensch und Computer 2021-Workshopband, 2021.

B. Laugwitz, T. Held, and M. Schrepp, “Construction and evaluation of a user experience questionnaire,” in Symposium of the Austrian HCI and usability engineering group, Springer, Berlin, Heidelberg, 2008, pp. 63-76.

B. Laugwitz, M. Schrepp, and T. Held, “Konstruktion eines Fragebogens zur Messung der User Experience von Softwareprodukten (Construction of a questionnaire to measure UX of software products),” in Mensch und Computer 2006, Oldenbourg Wissenschaftsverlag, 2006, pp. 125-134.

A. Hinderks, M. Schrepp, F. J. D. Mayo, M. J. Escalona, and J. Thomaschewski, “Developing a UX KPI based on the user experience questionnaire,“ Computer Standards & Interfaces, vol. 65, 2019 pp. 38-44.

M. Schrepp, A. Hinderks, and J. Thomaschewski, “Applying the user experience questionnaire (UEQ) in different evaluation scenarios,” in International Conference of Design, User Experience, and Usability, Springer, Cham, 2014, pp. 383-392.

M. Schrepp, A. Hinderks, and J. Thomaschewski, ”Construction of a benchmark for the User Experience Questionnaire (UEQ),” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 4, no. 4, 2017, pp. 40-44.

J. Brooke, “SUS-A quick and dirty usability scale,” Usability Evaluation in Industry, vol. 189, no. 194, 1996, pp. 4-7.

J. Kirakowski, and M. Corbett, “SUMI: The software usability measurement inventory,” British Journal of Educational Technology, vol. 24, no. 3, 1993, pp.210-212.

K. Finstad, “The usability metric for user experience,” Interacting with Computers, vol. 22, no. 5, 2010, pp. 323-327.

J. R. Lewis, B. Utesch, and D. E. Maher, D. E., “UMUX-LITE: when there’s no time for the SUS,” in Proceedings of the SIGCHI conference on human factors in computing systems, 2013, pp. 2099-2102.

J. Sauro, “SUPR-Q: A comprehensive measure of the quality of the website user experience,” Journal of usability studies, vol. 10, no. 2, 2015.

M. Moshagen, and M. Thielsch, “Facets of visual aesthetics,” International Journal of Human-Computer Studies, vol. 68, no. 10, 2010, pp. 689-709.

J. Sauro, and J. R. Lewis, Quantifying the user experience: Practical statistics for user research, 2nd ed., Cambridge, MA: Morgan-Kaufmann, 2016.

J. Brooke, “SUS A Retrospective,” Journal of Usability Studies, vol. 8, no. 2, 2013, pp. 29-40.

A. Bangor, P. T. Kortum, and J. T. Miller, “An empirical evaluation of the System Usability Scale,” International Journal of Human-Computer Interaction, vol. 24, no. 6, 2008, pp. 574-594.

A. Bangor, P. T. Kortum, and J. T. Miller, J. T., “Determining what individual SUS scores mean: Adding an adjective rating scale,” Journal of Usability Studies, vol. 4, no. 3, 2009 pp. 114-123.

M. Schrepp, S. Olschner, and U. Schubert, „User Experience Questionnaire (UEQ) Benchmark,” in Tagungsband UP13, 2013.

M. Moshagen and M. Thielsch, “A short version of the visual aesthetics of websites inventory,” Behaviour & Information Technology, vol. 32, no. 12, 2013, pp. 1305-1311.

A. Hinderks, F. J. Domínguez-Mayo, A.-L. Meiners, and J. Thomaschewski, “Applying Importance-Performance Analysis (IPA) to interpret the results of the User Experience Questionnaire (UEQ),” Journal of Web Engineering, vol. 19, no. 2, 2020, pp. 243-266.