Explainable AI, Academic Confidence, and Decision-Making Quality in Physics Learning
DOI:
10.70211/disolife.v2i1.777Published:
2026-06-30Downloads
Abstract
Opaque artificial intelligence (AI) recommendations may undermine learners' ability to evaluate feedback and make informed decisions. This study examined whether an explainable artificial intelligence (XAI) learning system was associated with higher academic confidence and decision-making quality than a comparable AI system without explanatory output during undergraduate physics learning. A quantitative quasi-experimental pretest-posttest control-group design involved 120 students (mean age = 20.3 years, SD = 1.4; 54.2% female), with 60 students analyzed in each condition across eight weeks and treatment implemented through three intact courses. Academic confidence was measured using an 18-item study-adapted Academic Confidence Scale-Short Form (alpha = .87), whereas decision-making quality was assessed using a 16-item Learning Task Decision-Making Inventory (alpha = .83). ANCOVA controlling for corresponding pretest scores showed higher posttest academic confidence in the XAI condition, F(1, 117) = 52.34, p < .001, partial eta squared = .309, and higher decision-making quality, F(1, 117) = 38.71, p < .001, partial eta squared = .249. The results support an association between the explanatory-interface condition and both learner outcomes, but they do not establish confidence calibration, deliberative processing, or the proposed transparency mechanism. Interpretation is constrained by course-level allocation across only three clusters, unmodeled course dependence, a single-institution sample, study-adapted questionnaire outcomes, and the absence of a manipulation check or explanation-fidelity evidence.
Keywords:
algorithmic transparency academic self-efficacy AI literacy higher education learner agencyReferences
[1] H. Crompton and D. Burke, “Artificial intelligence in higher education: The state of the field,” International Journal of Educational Technology in Higher Education, vol. 20, Art. no. 22, 2023. Available: https://doi.org/10.1186/s41239-023-00392-8 DOI: https://doi.org/10.1186/s41239-023-00392-8
[2] M. Bond, H. Khosravi, M. de Laat, N. Bergdahl, V. Negrea, E. Oxley, et al., “A meta systematic review of artificial intelligence in higher education: A call for increased ethics, collaboration, and rigour,” International Journal of Educational Technology in Higher Education, vol. 21, Art. no. 4, 2024. Available: https://doi.org/10.1186/s41239-023-00436-z DOI: https://doi.org/10.1186/s41239-023-00436-z
[3] O. Zawacki-Richter, J. Y. H. Bai, K. Lee, P. J. Slagter van Tryon, and P. Prinsloo, “New advances in artificial intelligence applications in higher education?,” International Journal of Educational Technology in Higher Education, vol. 21, Art. no. 32, 2024. Available: https://doi.org/10.1186/s41239-024-00464-3 DOI: https://doi.org/10.1186/s41239-024-00464-3
[4] C. K. Y. Chan, “A comprehensive AI policy education framework for university teaching and learning,” International Journal of Educational Technology in Higher Education, vol. 20, Art. no. 38, 2023. Available: https://doi.org/10.1186/s41239-023-00408-3 DOI: https://doi.org/10.1186/s41239-023-00408-3
[5] Y. An, J. H. Yu, and S. James, “Investigating the higher education institutions' guidelines and policies regarding the use of generative AI in teaching, learning, research, and administration,” International Journal of Educational Technology in Higher Education, vol. 22, Art. no. 10, 2025. Available: https://doi.org/10.1186/s41239-025-00507-3 DOI: https://doi.org/10.1186/s41239-025-00507-3
[6] C. K. Y. Chan and W. Hu, “Students' voices on generative AI: Perceptions, benefits, and challenges in higher education,” International Journal of Educational Technology in Higher Education, vol. 20, Art. no. 43, 2023. Available: https://doi.org/10.1186/s41239-023-00411-8 DOI: https://doi.org/10.1186/s41239-023-00411-8
[7] A. Yusuf, N. Pervin, and M. Román-González, “Generative AI and the future of higher education: A threat to academic integrity or reformation? Evidence from multicultural perspectives,” International Journal of Educational Technology in Higher Education, vol. 21, Art. no. 21, 2024. Available: https://doi.org/10.1186/s41239-024-00453-6 DOI: https://doi.org/10.1186/s41239-024-00453-6
[8] H. Jo, “From concerns to benefits: A comprehensive study of ChatGPT usage in education,” International Journal of Educational Technology in Higher Education, vol. 21, Art. no. 35, 2024. Available: https://doi.org/10.1186/s41239-024-00471-4 DOI: https://doi.org/10.1186/s41239-024-00471-4
[9] M. F. Shahzad, S. Xu, and I. Javed, “ChatGPT awareness, acceptance, and adoption in higher education: The role of trust as a cornerstone,” International Journal of Educational Technology in Higher Education, vol. 21, Art. no. 46, 2024. Available: https://doi.org/10.1186/s41239-024-00478-x DOI: https://doi.org/10.1186/s41239-024-00478-x
[10] H. Moradi, “Integrating AI in higher education: Factors influencing ChatGPT acceptance among Chinese university EFL students,” International Journal of Educational Technology in Higher Education, vol. 22, Art. no. 30, 2025. Available: https://doi.org/10.1186/s41239-025-00530-4 DOI: https://doi.org/10.1186/s41239-025-00530-4
[11] M. Abbas, F. A. Jam, and T. I. Khan, “Is it harmful or helpful? Examining the causes and consequences of generative AI usage among university students,” International Journal of Educational Technology in Higher Education, vol. 21, Art. no. 10, 2024. Available: https://doi.org/10.1186/s41239-024-00444-7 DOI: https://doi.org/10.1186/s41239-024-00444-7
[12] F. Wang, N. Li, A. C. K. Cheung, and G. K. W. Wong, “In GenAI we trust: An investigation of university students' reliance on and resistance to generative AI in language learning,” International Journal of Educational Technology in Higher Education, vol. 22, Art. no. 59, 2025. Available: https://doi.org/10.1186/s41239-025-00547-9 DOI: https://doi.org/10.1186/s41239-025-00547-9
[13] Y. Walter, “Embracing the future of artificial intelligence in the classroom: The relevance of AI literacy, prompt engineering, and critical thinking in modern education,” International Journal of Educational Technology in Higher Education, vol. 21, Art. no. 15, 2024. Available: https://doi.org/10.1186/s41239-024-00448-3 DOI: https://doi.org/10.1186/s41239-024-00448-3
[14] D. Lee and E. Palmer, “Prompt engineering in higher education: A systematic review to help inform curricula,” International Journal of Educational Technology in Higher Education, vol. 22, Art. no. 7, 2025. Available: https://doi.org/10.1186/s41239-025-00503-7 DOI: https://doi.org/10.1186/s41239-025-00503-7
[15] J. Delcker, J. Heil, D. Ifenthaler, S. Seufert, and L. Spirgi, “First-year students AI-competence as a predictor for intended and de facto use of AI-tools for supporting learning processes in higher education,” International Journal of Educational Technology in Higher Education, vol. 21, Art. no. 18, 2024. Available: https://doi.org/10.1186/s41239-024-00452-7 DOI: https://doi.org/10.1186/s41239-024-00452-7
[16] D.-K. Mah and N. Groß, “Artificial intelligence in higher education: Exploring faculty use, self-efficacy, distinct profiles, and professional development needs,” International Journal of Educational Technology in Higher Education, vol. 21, Art. no. 58, 2024. Available: https://doi.org/10.1186/s41239-024-00490-1 DOI: https://doi.org/10.1186/s41239-024-00490-1
[17] M. I. Baig and E. Yadegaridehkordi, “Factors influencing academic staff satisfaction and continuous usage of generative artificial intelligence (GenAI) in higher education,” International Journal of Educational Technology in Higher Education, vol. 22, Art. no. 5, 2025. Available: https://doi.org/10.1186/s41239-025-00506-4 DOI: https://doi.org/10.1186/s41239-025-00506-4
[18] J. Kamali, M. F. Alpat, and A. Bozkurt, “AI ethics as a complex and multifaceted challenge: Decoding educators' AI ethics alignment through the lens of activity theory,” International Journal of Educational Technology in Higher Education, vol. 21, Art. no. 62, 2024. Available: https://doi.org/10.1186/s41239-024-00496-9 DOI: https://doi.org/10.1186/s41239-024-00496-9
[19] T. Lim, S. Gottipati, and M. Cheong, “What students really think: Unpacking AI ethics in educational assessments through a triadic framework,” International Journal of Educational Technology in Higher Education, vol. 22, Art. no. 56, 2025. Available: https://doi.org/10.1186/s41239-025-00556-8 DOI: https://doi.org/10.1186/s41239-025-00556-8
[20] T. Yang, J. Cheon, M.-H. Cho, M. Huang, and N. Cusson, “Undergraduate students' perspectives of generative AI ethics,” International Journal of Educational Technology in Higher Education, vol. 22, Art. no. 35, 2025. Available: https://doi.org/10.1186/s41239-025-00533-1 DOI: https://doi.org/10.1186/s41239-025-00533-1
[21] S.-H. Jin, K. Im, M. Yoo, I. Roll, and K. Seo, “Supporting students' self-regulated learning in online learning using artificial intelligence applications,” International Journal of Educational Technology in Higher Education, vol. 20, Art. no. 37, 2023. Available: https://doi.org/10.1186/s41239-023-00406-5 DOI: https://doi.org/10.1186/s41239-023-00406-5
[22] J. Escalante, A. Pack, and A. Barrett, “AI-generated feedback on writing: Insights into efficacy and ENL student preference,” International Journal of Educational Technology in Higher Education, vol. 20, Art. no. 57, 2023. Available: https://doi.org/10.1186/s41239-023-00425-2 DOI: https://doi.org/10.1186/s41239-023-00425-2
[23] K. J. Topping, E. Gehringer, H. Khosravi, S. Gudipati, K. Jadhav, and S. Susarla, “Enhancing peer assessment with artificial intelligence,” International Journal of Educational Technology in Higher Education, vol. 22, Art. no. 3, 2025. Available: https://doi.org/10.1186/s41239-024-00501-1 DOI: https://doi.org/10.1186/s41239-024-00501-1
[24] H. Ates, “Human-centered GenAI feedback design in higher education: A multisite experiment on direct, reflective, and hybrid approaches to scientific argumentation,” International Journal of Educational Technology in Higher Education, vol. 23, Art. no. 38, 2026. Available: https://doi.org/10.1186/s41239-026-00614-9 DOI: https://doi.org/10.1186/s41239-026-00614-9
[25] Y. Hwang and J. H. Lee, “Exploring students' experiences and perceptions of human-AI collaboration in digital content making,” International Journal of Educational Technology in Higher Education, vol. 22, Art. no. 44, 2025. Available: https://doi.org/10.1186/s41239-025-00542-0 DOI: https://doi.org/10.1186/s41239-025-00542-0
[26] Y. Song, L. Huang, L. Zheng, M. Fan, and Z. Liu, “Interactions with generative AI chatbots: Unveiling dialogic dynamics, students' perceptions, and practical competencies in creative problem-solving,” International Journal of Educational Technology in Higher Education, vol. 22, Art. no. 12, 2025. Available: https://doi.org/10.1186/s41239-025-00508-2 DOI: https://doi.org/10.1186/s41239-025-00508-2
[27] M. Yilmaz, H. B. Temur, L. Emmungil, E. Çelik, A. Gauthier, and M. Cukurova, “Supporting self-regulated learning through generative AI feedback in online higher education: The importance of student perceptions of the source of feedback,” International Journal of Educational Technology in Higher Education, vol. 23, Art. no. 16, 2026. Available: https://doi.org/10.1186/s41239-026-00592-y DOI: https://doi.org/10.1186/s41239-026-00592-y
[28] L. Labadze, M. Grigolia, and L. Machaidze, “Role of AI chatbots in education: Systematic literature review,” International Journal of Educational Technology in Higher Education, vol. 20, Art. no. 56, 2023. Available: https://doi.org/10.1186/s41239-023-00426-1 DOI: https://doi.org/10.1186/s41239-023-00426-1
[29] A. Williams, “Comparison of generative AI performance on undergraduate and postgraduate written assessments in the biomedical sciences,” International Journal of Educational Technology in Higher Education, vol. 21, Art. no. 52, 2024. Available: https://doi.org/10.1186/s41239-024-00485-y DOI: https://doi.org/10.1186/s41239-024-00485-y
[30] K. Zdravkova and B. Ilijoski, “The impact of large language models on computer science student writing,” International Journal of Educational Technology in Higher Education, vol. 22, Art. no. 32, 2025. Available: https://doi.org/10.1186/s41239-025-00525-1 DOI: https://doi.org/10.1186/s41239-025-00525-1
[31] H. Khosravi, S. B. Shum, G. Chen, C. Conati, Y.-S. Tsai, J. Kay, et al., “Explainable artificial intelligence in education,” Computers and Education: Artificial Intelligence, vol. 3, Art. no. 100074, 2022. Available: https://doi.org/10.1016/j.caeai.2022.100074 DOI: https://doi.org/10.1016/j.caeai.2022.100074
[32] T. Susnjak, “Beyond predictive learning analytics modelling and onto explainable artificial intelligence with prescriptive analytics and ChatGPT,” International Journal of Artificial Intelligence in Education, vol. 34, pp. 452-482, 2024. Available: https://doi.org/10.1007/s40593-023-00336-3 DOI: https://doi.org/10.1007/s40593-023-00336-3
[33] Y. Feldman-Maggor, M. Cukurova, C. Kent, and G. Alexandron, “The impact of explainable AI on teachers' trust and acceptance of AI EdTech recommendations: The power of domain-specific explanations,” International Journal of Artificial Intelligence in Education, vol. 35, pp. 2889-2922, 2025. Available: https://doi.org/10.1007/s40593-025-00486-6 DOI: https://doi.org/10.1007/s40593-025-00486-6
[34] A. Bandura, “Self-efficacy: Toward a unifying theory of behavioral change,” Psychological Review, 84(2), 191-215, 1977. Available: https://doi.org/10.1037/0033-295X.84.2.191 DOI: https://doi.org/10.1037/0033-295X.84.2.191
[35] D. Kahneman, Thinking, fast and slow. Farrar, Straus and Giroux, 2011.
[36] Q. Xia, X. Weng, F. Ouyang, T.-J. Lin, and T. K. F. Chiu, “A scoping review on how generative artificial intelligence transforms assessment in higher education,” International Journal of Educational Technology in Higher Education, vol. 21, Art. no. 40, 2024. Available: https://doi.org/10.1186/s41239-024-00468-z DOI: https://doi.org/10.1186/s41239-024-00468-z
[37] R. Ellis, F. Han, and H. Cook, “Qualitatively different teacher experiences of teaching with generative artificial intelligence,” International Journal of Educational Technology in Higher Education, vol. 22, Art. no. 33, 2025. Available: https://doi.org/10.1186/s41239-025-00532-2 DOI: https://doi.org/10.1186/s41239-025-00532-2
[38] P. Wang and H. Ding, “The rationality of explanation or human capacity? Understanding the impact of explainable artificial intelligence on human-AI trust and decision performance,” Information Processing & Management, vol. 61, no. 4, Art. no. 103732, 2024. Available: https://doi.org/10.1016/j.ipm.2024.103732 DOI: https://doi.org/10.1016/j.ipm.2024.103732
[39] R. Fok and D. S. Weld, “In search of verifiability: Explanations rarely enable complementary performance in AI-advised decision making,” AI Magazine, vol. 45, no. 3, pp. 317-332, 2024. Available: https://doi.org/10.1002/aaai.12182 DOI: https://doi.org/10.1002/aaai.12182
[40] S. Pareek, N. van Berkel, E. Velloso, and J. Goncalves, “Effect of Explanation Conceptualisations on Trust in AI-assisted Credibility Assessment,” Proceedings of the ACM on Human-Computer Interaction, vol. 8, no. CSCW2, Art. no. 383, pp. 1-31, 2024. Available: https://doi.org/10.1145/3686922 DOI: https://doi.org/10.1145/3686922
[41] M. Naiseh, A. Simkute, B. Zieni, N. Jiang, and R. Ali, “C-XAI: A conceptual framework for designing XAI tools that support trust calibration,” Journal of Responsible Technology, vol. 17, Art. no. 100076, 2024. Available: https://doi.org/10.1016/j.jrt.2024.100076 DOI: https://doi.org/10.1016/j.jrt.2024.100076
[42] D. Lee, J. Pruitt, T. Zhou, J. Du, and B. Odegaard, “Metacognitive sensitivity: The key to calibrating trust and optimal decision making with AI,” PNAS Nexus, vol. 4, no. 5, Art. no. pgaf133, 2025. Available: https://doi.org/10.1093/pnasnexus/pgaf133 DOI: https://doi.org/10.1093/pnasnexus/pgaf133
License
Copyright (c) 2026 Nur Hidayah, Sodikin Sodikin
This work is licensed under a Creative Commons Attribution 4.0 International License.
.png)
