How Effective is Talkpal.ai in Enhancing English Proficiency? Insights from an Experimental Study
DOI:
10.70211/ltsm.v2i1.48Published:
2024-06-30Downloads
Abstract
Mastering English speaking skills is an essential competency for students of the English Education Study Program, particularly in facing the challenges of globalization. With the advancement of technology, Artificial Intelligence (AI) has emerged as a promising innovation to enhance the quality of language learning. This study aims to evaluate the effectiveness of Talkpal.ai, an AI-based application, in improving English speaking skills among second-semester students of the English Education Study Program. The research employs an experimental method with a pre-test and post-test control group design, involving 100 students randomly divided into two groups: an experimental group (n=50) using Talkpal.ai for 8 weeks, and a control group (n=50) learning through traditional teaching methods. Th e instrument used was a validated English speaking skills test, with data analysis conducted using a t-test. The results show a significant improvement in speaking skills among students in the experimental group (M=85, SD=5) compared to the control group (M=78, SD=7), with a t-value of t (98) = 5.47, p < 0.01. These findings indicate that Talkpal.ai is effective in providing a personalized and interactive learning experience, contributing to the enhancement of speaking skills. This study highlights the significant potential of AI technology in higher education, particularly in language learning, and offers recommendations for further implementation and future research.
Keywords:
Artificial Intelligence Educational Technology English Language Learning Talkpal.ai.References
I. Aizawa, H. Rose, G. Thompson, and S. Curle, "Beyond the threshold: Exploring English language proficiency, linguistic challenges, and academic language skills of Japanese students in an English medium instruction programme," Lang. Teach. Res., vol. 27, no. 4, pp. 837-861, Jul. 2023. https://doi.org/10.1177/1362168820965510
L. Wei, "Translanguaging as a political stance: implications for English language education," ELT J., vol. 76, no. 2, pp. 172-182, Apr. 2022. https://doi.org/10.1093/elt/ccab083
H. Guo and W. Gao, "Metaverse-Powered Experiential Situational English-Teaching Design: An Emotion-Based Analysis Method," Front. Psychol., vol. 13, Mar. 2022. https://doi.org/10.3389/fpsyg.2022.859159
J. M. Gayed, M. K. J. Carlon, A. M. Oriola, and J. S. Cross, "Exploring an AI-based writing Assistant's impact on English language learners," Comput. Educ. Artif. Intell., vol. 3, p. 100055, 2022. https://doi.org/10.1016/j.caeai.2022.100055
W. Xie and S. Curle, "Success in English Medium Instruction in China: significant indicators and implications," Int. J. Biling. Educ. Biling., vol. 25, no. 2, pp. 585-597, Feb. 2022. https://doi.org/10.1080/13670050.2019.1703898
D. Block and B. Moncada-Comas, "English-medium instruction in higher education and the ELT gaze: STEM lecturers' self-positioning as NOT English language teachers," Int. J. Biling. Educ. Biling., vol. 25, no. 2, pp. 401-417, Feb. 2022. https://doi.org/10.1080/13670050.2019.1689917
X. Hu, X. Zhang, and S. McGeown, "Foreign language anxiety and achievement: A study of primary school students learning English in China," Lang. Teach. Res., vol. 28, no. 4, pp. 1594-1615, Jul. 2024. https://doi.org/10.1177/13621688211032332
T. K. A. Dang, G. Bonar, and J. Yao, "Professional learning for educators teaching in English-medium-instruction in higher education: a systematic review," Teach. High. Educ., vol. 28, no. 4, pp. 840-858, May 2023. https://doi.org/10.1080/13562517.2020.1863350
M. Li and Z. Yu, "A systematic review on the metaverse-based blended English learning," Front. Psychol., vol. 13, Jan. 2023. https://doi.org/10.3389/fpsyg.2022.1087508
Kaharuddin, D. Ahmad, Mardiana, I. Latif, B. Arafah, and R. Suryadi, "Defining the Role of Artificial Intelligence in Improving English Writing Skills Among Indonesian Students," J. Lang. Teach. Res., vol. 15, no. 2, pp. 568-678, Mar. 2024. https://doi.org/10.17507/jltr.1502.25
A. Isma, P. W. Sudewi, and A. Amrang, "Exploring the Attitudes of English Language Learners toward Online Learning in Indonesian Higher Education," J. Asiat., vol. 21, no. 1, pp. 207-216, Mar. 2024. https://doi.org/10.18823/asiatefl.2024.21.1.13.207
I. Mutiaraningrum, S. W. Fitriati, I. Yuliasri, and M. Saleh, "Indonesian vocational college students' attitudes towards project-based learning in English courses," Int. J. Eval. Res. Educ., vol. 13, no. 5, p. 3177, Oct. 2024. https://doi.org/10.11591/ijere.v13i5.28406
H. Usman, I. Lestari, Y. E. Y. Siregar, S. R. Rafiqa, and I. Sentryo, "Flipbook and e-learning for teaching English to elementary school teacher education students," Stud. English Lang. Educ., vol. 11, no. 2, pp. 919-935, Jun. 2024. https://doi.org/10.24815/siele.v11i2.35476
Noprival and Alfian, "Language learning strategies used by Indonesian English for medical purposes students in higher education," Learn. Res. Pract., pp. 1-14, Feb. 2024. https://doi.org/10.1080/23735082.2024.2317829
A. Strzelecki, "To use or not to use ChatGPT in higher education? A study of students' acceptance and use of technology," Interact. Learn. Environ., pp. 1-14, May 2023. https://doi.org/10.1080/10494820.2023.2209881
C. A. Ongoro and Y.-Y. Fanjiang, "Digital Game-Based Technology for English Language Learning in Preschools and Primary Schools: A Systematic Analysis," IEEE Trans. Learn. Technol., vol. 17, pp. 202-228, 2024. https://doi.org/10.1109/TLT.2023.3268282
M. H. Al-khresheh, "Bridging technology and pedagogy from a global lens: Teachers' perspectives on integrating ChatGPT in English language teaching," Comput. Educ. Artif. Intell., vol. 6, p. 100218, Jun. 2024. https://doi.org/10.1016/j.caeai.2024.100218
T. J. Allen and A. Mizumoto, "ChatGPT Over My Friends: Japanese English-as-a-Foreign-Language Learners' Preferences for Editing and Proofreading Strategies," RELC J., Jul. 2024. https://doi.org/10.1177/00336882241262533
M. F. Teng, "A Systematic Review of ChatGPT for English as a Foreign Language Writing: Opportunities, Challenges, and Recommendations," Int. J. TESOL Stud., vol. 6, no. 3, pp. 36-57, Jul. 2024. https://doi.org/10.58304/ijts.20240304
C. Zhai and S. Wibowo, "A systematic review on artificial intelligence dialogue systems for enhancing English as foreign language students' interactional competence in the university," Comput. Educ. Artif. Intell., vol. 4, p. 100134, 2023. https://doi.org/10.1016/j.caeai.2023.100134
N. Hockly, "Artificial Intelligence in English Language Teaching: The Good, the Bad and the Ugly," RELC J., vol. 54, no. 2, pp. 445-451, Aug. 2023. https://doi.org/10.1177/00336882231168504
S. Shaikh, S. Y. Yayilgan, B. Klimova, and M. Pikhart, "Assessing the Usability of ChatGPT for Formal English Language Learning," Eur. J. Investig. Heal. Psychol. Educ., vol. 13, no. 9, pp. 1937-1960, Sep. 2023. https://doi.org/10.3390/ejihpe13090140
J. C. Young and M. Shishido, "Investigating OpenAI's ChatGPT Potentials in Generating Chatbot's Dialogue for English as a Foreign Language Learning," Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 6, 2023. https://doi.org/10.14569/IJACSA.2023.0140607
L. Wei, "Artificial intelligence in language instruction: impact on English learning achievement, L2 motivation, and self-regulated learning," Front. Psychol., vol. 14, Nov. 2023. https://doi.org/10.3389/fpsyg.2023.1261955
J. H. Lee, D. Shin, and W. Noh, "Artificial Intelligence-Based Content Generator Technology for Young English-as-a-Foreign-Language Learners' Reading Enjoyment," RELC J., vol. 54, no. 2, pp. 508-516, Aug. 2023. https://doi.org/10.1177/00336882231165060
Efendi Hidayatullah, "The Impact of Talkpal.AI on English Speaking Proficiency: An Academic Inquiry," J. Insa. Mulia Educ., vol. 2, no. 1, pp. 19-25, Apr. 2024. https://doi.org/10.59923/joinme.v2i1.98
Q. Lyu et al., "Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential," Vis. Comput. Ind. Biomed. Art, vol. 6, no. 1, p. 9, May 2023. https://doi.org/10.1186/s42492-023-00136-5
L. Krstić, V. Aleksić, and M. Krstić, "Artificial Intelligence in Education: A Review," in Proceedings TIE 2022, University of Kragujevac, Faculty of Technical Sciences Čačak, 2022, pp. 223-228. https://doi.org/10.46793/TIE22.223K
J. Belda-Medina and J. R. Calvo-Ferrer, "Using Chatbots as AI Conversational Partners in Language Learning," Appl. Sci., vol. 12, no. 17, p. 8427, Aug. 2022. https://doi.org/10.3390/app12178427
K. Mageira, D. Pittou, A. Papasalouros, K. Kotis, P. Zangogianni, and A. Daradoumis, "Educational AI Chatbots for Content and Language Integrated Learning," Appl. Sci., vol. 12, no. 7, p. 3239, Mar. 2022. https://doi.org/10.3390/app12073239
Z. Ismail, N. Rasit, and T. Supriyatno, "Relationship Between Oral Language Anxiety and Students' Arabic Language Learning Outcomes in Malaysian Secondary Schools," Int. J. Lang. Educ., vol. 1, no. 1, p. 143, Mar. 2023. https://doi.org/10.26858/ijole.v1i1.37368
N. Ivanova, V. Gugleva, M. Dobreva, I. Pehlivanov, S. Stefanov, and V. Andonova, Human capital in the smart manufacturing and industry 4.0 revolution. Web of Science, 2018.
F. Pikri, "The Role of the Language Environment in Improving Arabic Learning Abilities," Int. J. Sci. Soc., vol. 4, no. 2, pp. 346-354, Jul. 2022. https://doi.org/10.54783/ijsoc.v4i2.478
M. Ritonga, S. R. Febriani, M. Kustati, E. Khaef, A. W. Ritonga, and R. Yasmar, "Duolingo: An Arabic Speaking Skills' Learning Platform for Andragogy Education," Educ. Res. Int., vol. 2022, pp. 1-9, Feb. 2022. https://doi.org/10.1155/2022/7090752
G. I. W. Tamtama, P. Suryanto, and S. Suyoto, "Design of English Vocabulary Mobile Apps Using Gamification: An Indonesian Case Study for Kindergarten," Int. J. Eng. Pedagog., vol. 10, no. 1, p. 150, Jan. 2020. https://doi.org/10.3991/ijep.v10i1.11551
L. Septiyana, A. Aneka, B. Mandasari, and F. S. S. Ramadhan, "Development of A Trilingual E-Dictionary for Early Childhood; Indonesia-English-Lampung," Tapis J. Penelit. Ilm., vol. 7, no. 2, p. 160, Sep. 2023. https://doi.org/10.32332/tapis.v7i2.7817
Author Biographies
Vigo Dikaprio, Universitas Sriwijaya
Department of English Language Education
.png)
Chuzaimah Dahlan Diem, Universitas Sriwijaya
Department of English Language Education
License
Copyright (c) 2024 Vigo Dikaprio, Chuzaimah Dahlan Diem
This work is licensed under a Creative Commons Attribution 4.0 International License.
