Does Chatting with Chatbots Improve Language Learning Performance? A Meta-Analysis of Chatbot-Assisted Language Learning
Saved in:
| Title: | Does Chatting with Chatbots Improve Language Learning Performance? A Meta-Analysis of Chatbot-Assisted Language Learning |
|---|---|
| Language: | English |
| Authors: | Feifei Wang, Alan C. K. Cheung, Amanda J. Neitzel (ORCID |
| Source: | Review of Educational Research. 2025 95(4):623-660. |
| Availability: | SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com |
| Peer Reviewed: | Y |
| Page Count: | 38 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Information Analyses Reports - Research |
| Education Level: | Elementary Secondary Education Adult Education |
| Descriptors: | Artificial Intelligence, Man Machine Systems, Natural Language Processing, Second Language Learning, Native Language, Language Acquisition, Language Proficiency, Computer Assisted Instruction, Elementary Secondary Education, Adult Education, Technology Uses in Education |
| DOI: | 10.3102/00346543241255621 |
| ISSN: | 0034-6543 1935-1046 |
| Abstract: | Given the importance of conversation practice in language learning, chatbots, especially ChatGPT, have attracted considerable attention for their ability to converse with learners using natural language. This review contributes to the literature by examining the currently unclear overall effect of using chatbots on language learning performance and comprehensively identifying important study characteristics that affect the overall effectiveness. We meta-analyzed 70 effect sizes from 28 studies, using robust variance estimation. The effects were assessed based on 18 study characteristics about learners, chatbots, learning objectives, context, communication/interaction, and methodological and pedagogical designs. Results indicated that using chatbots produced a positive overall effect on language learning performance (g = 0.484), compared to nonchatbot conditions. Moreover, four characteristics (i.e., educational level, language level, interface design, and interaction capability) affected the overall effectiveness. In an in-depth discussion on how the 18 characteristics are related to the effectiveness, future implications for practice and research are presented. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1476911 |
| Database: | ERIC |
| Abstract: | Given the importance of conversation practice in language learning, chatbots, especially ChatGPT, have attracted considerable attention for their ability to converse with learners using natural language. This review contributes to the literature by examining the currently unclear overall effect of using chatbots on language learning performance and comprehensively identifying important study characteristics that affect the overall effectiveness. We meta-analyzed 70 effect sizes from 28 studies, using robust variance estimation. The effects were assessed based on 18 study characteristics about learners, chatbots, learning objectives, context, communication/interaction, and methodological and pedagogical designs. Results indicated that using chatbots produced a positive overall effect on language learning performance (g = 0.484), compared to nonchatbot conditions. Moreover, four characteristics (i.e., educational level, language level, interface design, and interaction capability) affected the overall effectiveness. In an in-depth discussion on how the 18 characteristics are related to the effectiveness, future implications for practice and research are presented. |
|---|---|
| ISSN: | 0034-6543 1935-1046 |
| DOI: | 10.3102/00346543241255621 |
