Comparing User Perceptions of User Interface and User Experience in Learning Management Systems: A UEQ-Based Study Across Two Universities
DOI:
https://doi.org/10.21831/elinvo.v11i1.86000Keywords:
Learning Management System, Equivalence Test, Paired t-test, Wilcoxon, UEQ, Statistical Testing, Statistical PowerAbstract
User perceptions toward software application interface and experience are important factors for the successful implementation of software systems. Therefore, this study was aiming to investigate these success factors by comparing user perceptions towards software application interface and experience when using software applications. This research used a inferential quantitative approach. The instrument used in this study was the UEQ (User Experience Questionnaire) formulated by the UEQ Team. The object of this study was two e-learning systems deployed at two state universities in Yogyakarta. Data on user perceptions toward software application interface and experience were collected twice. The collected data were then statistically analysed to obtain results for equivalence test, paired t-tests, Wilcoxon tests, statistical power, and Cohen's D values. Based on equivalence testing on the data gathered during the first and second collection stages, it was found that user perceptions toward software application interface and experience were equivalent and not different for most criteria and sub-criteria, except for the enjoyable, good, pleasing, and pleasant sub-criteria at University A and enjoyable, good, and motivating at University B. Meanwhile, based on the results of paired t-tests and Wilcoxon tests, it was found that user perceptions toward software application interface and experience were also equivalent and not different for most criteria and sub-criteria, except for the pleasing and good sub-criteria at University A, and the enjoyable and pleasing sub-criteria at University B. Therefore, it is concluded that there was generally no difference in user perceptions toward software application interface and experience when using those e-learning systems.
References
[1] H. Kivijärvi and K. Pärnänen, “Instrumental Usability and Effective User Experience: Interwoven Drivers and Outcomes of Human-Computer Interaction,” Int. J. Human–Computer Interact., vol. 39, no. 1, pp. 34–51, Jan. 2023, doi: 10.1080/10447318.2021.2016236.
[2] I. F. B. Andoro, “Analisis Tingkat Usability dan User Experience LMS Institut Widya Pratama dengan Pendekatan UEQ (User Experience Questionnaire),” J. Ilm. Sist. Inf. dan Ilmu Komput., 2025, doi: 10.55606/juisik.v5i1.1471.
[3] P. R et al., “Human-Computer Interaction: Enhancing User Experience in Interactive Systems,” in E3S Web of Conferences, 2023. doi: 10.1051/e3sconf/202339904037.
[4] F. Fitria, M. Yahya, M. Ali, P. Purnamawati, and A. M. Mappalotteng, “The Impact of System Quality and User Satisfaction: The Mediating Role of Ease of Use and Usefulness in E-Learning Systems,” Int. J. Environ. Eng. Educ., 2024, doi: 10.55151/ijeedu.v6i2.134.
[5] H.-S. Wong and H. Ip, “Human Computer Interaction,” pp. 289–293, 2015, doi: 10.1007/978-3-319-03068-5.
[6] J. Kollmorgen, A. Hinderks, and J. Thomaschewski, “Selecting the Appropriate User Experience Questionnaire and Guidance for Interpretation: the UEQ Family,” Int. J. Interact. Multim. Artif. Intell., vol. 9, pp. 126–139, 2024, doi: 10.9781/ijimai.2024.08.005.
[7] M. Maguire, “Using Human Factors Standards to Support User Experience and Agile Design,” pp. 185–194, 2013, doi: 10.1007/978-3-642-39188-0_20.
[8] R. Juárez-Ramírez, “User-centered design and adaptive systems: toward improving usability and accessibility,” Univers. Access Inf. Soc., vol. 16, pp. 361–363, 2017, doi: 10.1007/s10209-016-0480-1.
[9] P. Vlachogianni and N. Tselios, “Perceived usability evaluation of educational technology using the System Usability Scale (SUS): A systematic review,” J. Res. Technol. Educ., vol. 54, pp. 392–409, 2021, doi: 10.1080/15391523.2020.1867938.
[10] A. Schankin, M. Budde, T. Riedel, and M. Beigl, “Psychometric Properties of the User Experience Questionnaire (UEQ),” in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 2022. doi: 10.1145/3491102.3502098.
[11] A. Saleh, H. Y. Abuaddous, I. Alansari, and O. Enaizan, “The Evaluation of User Experience on Learning Management Systems Using UEQ,” Int. J. Emerg. Technol. Learn., vol. 17, pp. 145–162, 2022, doi: 10.3991/ijet.v17i07.29525.
[12] D. Quiñones and L. Rojas, “Understanding the customer experience in human-computer interaction: a systematic literature review,” PeerJ Comput. Sci., vol. 9, 2023, doi: 10.7717/peerj-cs.1219.
[13] R. K. Paredes and A. A. Hernandez, “Measuring the quality of user experience on web services: A case of university in the Philippines,” in 2017IEEE 9th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management (HNICEM), 2017, pp. 1–6. doi: 10.1109/HNICEM.2017.8269446.
[14] W. M. Lim, “What Is Quantitative Research? An Overview and Guidelines,” Australas. Mark. J., vol. 33, pp. 325–348, 2024, doi: 10.1177/14413582241264622.
[15] G. Kotronoulas et al., “An Overview of the Fundamentals of Data Management, Analysis, and Interpretation in Quantitative Research.,” Semin. Oncol. Nurs., p. 151398, 2023, doi: 10.1016/j.soncn.2023.151398.
[16] H. Mohajan, “Quantitative Research: A Successful Investigation in Natural and Social Sciences,” J. Econ. Dev. Environ. People, 2020, doi: 10.26458/jedep.v9i4.679.
[17] D. Chawla and A. Deorari, “Inferential Statistics: Introduction to Hypothesis Testing,” J. Neonatol., vol. 19, pp. 259–264, 2005, doi: 10.1177/0973217920050313.
[18] R. Wilcox, “Inferential Statistics,” Oxford Res. Encycl. Bus. Manag., 2021, doi: 10.1093/acrefore/9780190224851.013.280.
[19] A. Kenny, “From Sample to Population,” pp. 89–104, 1982, doi: 10.1016/b978-0-08-024281-1.50012-7.
[20] S. R. Henim and R. P. Sari, “User Experience Evaluation of Student Academic Information System of Higher Education Using User Experience Questionnaire,” J. Komput. Terap., 2020, doi: 10.35143/jkt.v6i1.3582.
[21] M. Schrepp, A. Hinderks, and J. Thomaschewski, “Design and Evaluation of a Short Version of the User Experience Questionnaire (UEQ-S),” Int. J. Interact. Multim. Artif. Intell., vol. 4, pp. 103–108, 2017, doi: 10.9781/ijimai.2017.09.001.
[22] M. Hassenzahl, “The Effect of Perceived Hedonic Quality on Product Appealingness,” Int. J. Human–Computer Interact., vol. 13, pp. 481–499, 2001, doi: 10.1207/s15327590ijhc1304_07.
[23] D. S. S. Sahid, P. I. Santosa, R. Ferdiana, and E. N. Lukito, “Evaluation and measurement of Learning Management System based on user experience,” in 2016 6th International Annual Engineering Seminar (InAES), 2016, pp. 72–77. doi: 10.1109/INAES.2016.7821910.
[24] M. Schrepp, A. Hinderks, and J. Thomaschewski, “Construction of a Benchmark for the User Experience Questionnaire (UEQ),” Int. J. Interact. Multim. Artif. Intell., vol. 4, pp. 40–44, 2017, doi: 10.9781/ijimai.2017.445.
[25] M. Rauschenberger, M. Schrepp, M. Cota, S. Olschner, and J. Thomaschewski, “Efficient Measurement of the User Experience of Interactive Products. How to use the User Experience Questionnaire (UEQ).Example: Spanish Language Version,” Int. J. Interact. Multim. Artif. Intell., vol. 2, pp. 39–45, 2013, doi: 10.9781/ijimai.2013.215.
[26] B. Laugwitz, T. Held, and M. Schrepp, “Construction and Evaluation of a User Experience Questionnaire,” pp. 63–76, 2008, doi: 10.1007/978-3-540-89350-9_6.
[27] M. Kankaraš and S. Capecchi, “Neither agree nor disagree: use and misuse of the neutral response category in Likert-type scales,” METRON, vol. 83, pp. 111–140, 2024, doi: 10.1007/s40300-024-00276-5.
[28] B. Zhang, J. Luo, and J. Li, “Moving beyond Likert and Traditional Forced-Choice Scales: A Comprehensive Investigation of the Graded Forced-Choice Format,” Multivariate Behav. Res., vol. 59, pp. 434–460, 2023, doi: 10.1080/00273171.2023.2235682.
[29] J. Nadler, R. Weston, and E. Voyles, “Stuck in the Middle: The Use and Interpretation of Mid-Points in Items on Questionnaires,” J. Gen. Psychol., vol. 142, pp. 71–89, 2015, doi: 10.1080/00221309.2014.994590.
[30] K. Kurtulus, C. Basfirinci, and Z. C. Uk, “Beyond the middle: rethinking midpoint and ‘no idea’ options in Likert scale for consumer sensitive issues,” Humanit. Soc. Sci. Commun., 2026, doi: 10.1057/s41599-026-07037-x.
[31] A. Adam, “A Study on Sample Size Determination in Survey Research,” pp. 125–134, 2021, doi: 10.9734/bpi/nicst/v4/2125e.
[32] A. Adam, “Sample Size Determination in Survey Research,” J. Sci. Res. Reports, pp. 90–97, 2020, doi: 10.9734/jsrr/2020/v26i530263.
[33] B. Mukti, “Methods in health research: Probability and non-probability sampling,” Heal. Sci. Int. J., 2025, doi: 10.71357/hsij.v3i2.64.
[34] S. Singh, “Simple Random Sampling,” pp. 71–136, 2003, doi: 10.1007/978-94-007-0789-4_2.
[35] S. H. P. W. Gamage, J. R. Ayres, and M. B. Behrend, “A systematic review on trends in using Moodle for teaching and learning,” Int. J. STEM Educ., vol. 9, no. 1, p. 9, 2022, doi: 10.1186/s40594-021-00323-x.
[36] R. Kelter, “Bayesian Hodges-Lehmann tests for statistical equivalence in the two-sample setting: Power analysis, type I error rates and equivalence boundary selection in biomedical research,” BMC Med. Res. Methodol., vol. 21, 2021, doi: 10.1186/s12874-021-01341-7.
[37] D. Lakens, A. Scheel, and P. Isager, “Equivalence Testing for Psychological Research: A Tutorial,” Adv. Methods Pract. Psychol. Sci., vol. 1, pp. 259–269, 2018, doi: 10.1177/2515245918770963.
[38] P. Riesthuis, “Simulation-Based Power Analyses for the Smallest Effect Size of Interest: A Confidence-Interval Approach for Minimum-Effect and Equivalence Testing,” Adv. Methods Pract. Psychol. Sci., vol. 7, 2024, doi: 10.1177/25152459241240722.
[39] R. Tapio, “The Role of Data Assumptions in Selecting Between Parametric and Nonparametric Tests,” Asian J. Probab. Stat., 2025, doi: 10.9734/ajpas/2025/v27i11830.
[40] J. A. Montero, “A Structured Guide to Univariate Test Selection Based on Normality, Variance Homogeneity, and Graphical Data Exploration.,” J. Surg. Res., vol. 318, pp. 230–240, 2026, doi: 10.1016/j.jss.2025.10.053.
[41] G. Ghadhban and H. Rasheed, “Robust Tests for the Mean Difference in Paired Data using Jackknife Resampling Technique,” Iraqi J. Sci., 2021, doi: 10.24996/ijs.2021.62.9.23.
[42] U. Tonggumnead, “A COMPARATIVE STUDY ON THE EFFICIENCY OF TEST STATISTICS IN TESTING THE DIFFERENCES BETWEEN TWO DEPENDENT DATASETS,” Int. J. Apllied Math., 2021, doi: 10.12732/ijam.v34i1.2.
[43] S. Dankel and J. Loenneke, “Effect Sizes for Paired Data Should Use the Change Score Variability Rather Than the Pre-test Variability,” J. Strength Cond. Res., vol. 35, pp. 1773–1778, 2018, doi: 10.1519/jsc.0000000000002946.
[44] B. Langenberg, M. Janczyk, V. Koob, R. Kliegl, and A. Mayer, “A tutorial on using the paired t test for power calculations in repeated measures ANOVA with interactions,” Behav. Res. Methods, vol. 55, pp. 2467–2484, 2022, doi: 10.3758/s13428-022-01902-8.
[45] H. Kang, “Sample size determination and power analysis using the G*Power software,” J. Educ. Eval. Health Prof., vol. 18, 2021, doi: 10.3352/jeehp.2021.18.17.
[46] B. Gerald and T. F. Patson, “Parametric and Nonparametric Tests: A Brief Review,” Int. J. Stat. Distrib. Appl., 2021, doi: 10.11648/j.ijsd.20210703.12.
[47] Budiono and A. Prasetia, “STUDI PERBANDINGAN HASIL UJI WILCOXON PADA DATA HASIL PENGUKURAN DAN HASIL KATEGORI DATA PENELITIAN KESEHATAN TINGKAT STRESS TEKANAN DARAH DAN MOTORIK HALUS,” J. Ilm. Pamenang, vol. 4, no. 2, pp. 8–15, 2022, doi: https://doi.org/10.53599/jip.v4i2.94.
[48] F. Fadilatunnisyah, R. F. S, E. A. Fasha, A. K. Putri, and D. A. J. D. Putri, “Penggunaan Uji Wilcoxon Signed Rank Test untuk Menganalisis Pengaruh Tingkat Motivasi Belajar Sebelum dan Sesudah Diterima di Universitas Impian,” Indones. J. Educ. Dev. Res., vol. 2, no. 1, 2024, doi: https://doi.org/10.57235/ijedr.v2i1.1887.
Downloads
Published
How to Cite
Issue
Section
Citation Check
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
The article published in ELINVO became ELINVO's right in publication.
This work by ELINVO is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
