View in EDS

Does Chatting with Chatbots Improve Language Learning Performance? A Meta-Analysis of Chatbot-Assisted Language Learning

Saved in:
Bibliographic Details
Title: Does Chatting with Chatbots Improve Language Learning Performance? A Meta-Analysis of Chatbot-Assisted Language Learning
Language: English
Authors: Feifei Wang, Alan C. K. Cheung, Amanda J. Neitzel (ORCID 0000-0002-4676-9320), Ching Sing Chai
Source: Review of Educational Research. 2025 95(4):623-660.
Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Peer Reviewed: Y
Page Count: 38
Publication Date: 2025
Document Type: Journal Articles
Information Analyses
Reports - Research
Education Level: Elementary Secondary Education
Adult Education
Descriptors: Artificial Intelligence, Man Machine Systems, Natural Language Processing, Second Language Learning, Native Language, Language Acquisition, Language Proficiency, Computer Assisted Instruction, Elementary Secondary Education, Adult Education, Technology Uses in Education
DOI: 10.3102/00346543241255621
ISSN: 0034-6543
1935-1046
Abstract: Given the importance of conversation practice in language learning, chatbots, especially ChatGPT, have attracted considerable attention for their ability to converse with learners using natural language. This review contributes to the literature by examining the currently unclear overall effect of using chatbots on language learning performance and comprehensively identifying important study characteristics that affect the overall effectiveness. We meta-analyzed 70 effect sizes from 28 studies, using robust variance estimation. The effects were assessed based on 18 study characteristics about learners, chatbots, learning objectives, context, communication/interaction, and methodological and pedagogical designs. Results indicated that using chatbots produced a positive overall effect on language learning performance (g = 0.484), compared to nonchatbot conditions. Moreover, four characteristics (i.e., educational level, language level, interface design, and interaction capability) affected the overall effectiveness. In an in-depth discussion on how the 18 characteristics are related to the effectiveness, future implications for practice and research are presented.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1476911
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1476911
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Does Chatting with Chatbots Improve Language Learning Performance? A Meta-Analysis of Chatbot-Assisted Language Learning
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Feifei+Wang%22">Feifei Wang</searchLink><br /><searchLink fieldCode="AR" term="%22Alan+C%2E+K%2E+Cheung%22">Alan C. K. Cheung</searchLink><br /><searchLink fieldCode="AR" term="%22Amanda+J%2E+Neitzel%22">Amanda J. Neitzel</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-4676-9320">0000-0002-4676-9320</externalLink>)<br /><searchLink fieldCode="AR" term="%22Ching+Sing+Chai%22">Ching Sing Chai</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Review+of+Educational+Research%22"><i>Review of Educational Research</i></searchLink>. 2025 95(4):623-660.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 38
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Information Analyses<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Adult+Education%22">Adult Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Man+Machine+Systems%22">Man Machine Systems</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Second+Language+Learning%22">Second Language Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Native+Language%22">Native Language</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Acquisition%22">Language Acquisition</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Proficiency%22">Language Proficiency</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Instruction%22">Computer Assisted Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink><br /><searchLink fieldCode="DE" term="%22Adult+Education%22">Adult Education</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.3102/00346543241255621
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0034-6543<br />1935-1046
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Given the importance of conversation practice in language learning, chatbots, especially ChatGPT, have attracted considerable attention for their ability to converse with learners using natural language. This review contributes to the literature by examining the currently unclear overall effect of using chatbots on language learning performance and comprehensively identifying important study characteristics that affect the overall effectiveness. We meta-analyzed 70 effect sizes from 28 studies, using robust variance estimation. The effects were assessed based on 18 study characteristics about learners, chatbots, learning objectives, context, communication/interaction, and methodological and pedagogical designs. Results indicated that using chatbots produced a positive overall effect on language learning performance (g = 0.484), compared to nonchatbot conditions. Moreover, four characteristics (i.e., educational level, language level, interface design, and interaction capability) affected the overall effectiveness. In an in-depth discussion on how the 18 characteristics are related to the effectiveness, future implications for practice and research are presented.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1476911
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1476911
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.3102/00346543241255621
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 38
        StartPage: 623
    Subjects:
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Man Machine Systems
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Second Language Learning
        Type: general
      – SubjectFull: Native Language
        Type: general
      – SubjectFull: Language Acquisition
        Type: general
      – SubjectFull: Language Proficiency
        Type: general
      – SubjectFull: Computer Assisted Instruction
        Type: general
      – SubjectFull: Elementary Secondary Education
        Type: general
      – SubjectFull: Adult Education
        Type: general
      – SubjectFull: Technology Uses in Education
        Type: general
    Titles:
      – TitleFull: Does Chatting with Chatbots Improve Language Learning Performance? A Meta-Analysis of Chatbot-Assisted Language Learning
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Feifei Wang
      – PersonEntity:
          Name:
            NameFull: Alan C. K. Cheung
      – PersonEntity:
          Name:
            NameFull: Amanda J. Neitzel
      – PersonEntity:
          Name:
            NameFull: Ching Sing Chai
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0034-6543
            – Type: issn-electronic
              Value: 1935-1046
          Numbering:
            – Type: volume
              Value: 95
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Review of Educational Research
              Type: main
ResultId 1