Effectiveness of Generative AI in Automated Written Corrective Feedback with Prompting
Saved in:
| Title: | Effectiveness of Generative AI in Automated Written Corrective Feedback with Prompting |
|---|---|
| Language: | English |
| Authors: | Jiahui Wu (ORCID |
| Source: | Journal of Educational Computing Research. 2025 63(6):1493-1527. |
| Availability: | SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com |
| Peer Reviewed: | Y |
| Page Count: | 35 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Elementary Education Secondary Education |
| Descriptors: | Error Correction, Feedback (Response), Artificial Intelligence, Computer Software, Technology Integration, Accuracy, Prompting, Writing Evaluation, Teacher Attitudes, Student Attitudes, Instructional Effectiveness, Self Efficacy, Faculty Workload, Educational Benefits, Writing Instruction, Elementary School Students, Secondary School Students, English (Second Language), Second Language Instruction, Second Language Learning, Foreign Countries |
| Geographic Terms: | China |
| DOI: | 10.1177/07356331251359430 |
| ISSN: | 0735-6331 1541-4140 |
| Abstract: | Automated written corrective feedback (AWCF) tools play a crucial role in supporting English writing instruction. However, issues such as insufficient accuracy and hallucination have undermined users' trust in these systems. To address these challenges, this study investigates the potential of Generative Artificial Intelligence (GAI) enhanced by prompting. Specifically, we evaluate the performance of several GAI models, including GPT-4, in providing written corrective feedback compared to commercial AWCF tools and other advanced models. The study adopts a dual-method approach: (1) a comprehensive model evaluation using established English writing datasets to assess error correction performance via various prompting strategies, and (2) an empirical study involving teachers and students to examine the system's practical efficacy and users' perspective. Quantitative results indicate that GPT-4 with chain-of-thought prompting significantly outperforms commercial tools, achieving improved consistency and accuracy in error detection and correction. Qualitative feedback from participants further supports the system's potential to enhance students' writing quality and confidence, while concurrently reducing teachers' workload and optimizing instructional efficiency, despite concerns regarding occasional overcorrection. These findings emphasize the benefits of task-specific prompting in GAI-based AWCF systems and provide actionable insights for integrating advanced AI feedback into educational practices. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1480245 |
| Database: | ERIC |
| Abstract: | Automated written corrective feedback (AWCF) tools play a crucial role in supporting English writing instruction. However, issues such as insufficient accuracy and hallucination have undermined users' trust in these systems. To address these challenges, this study investigates the potential of Generative Artificial Intelligence (GAI) enhanced by prompting. Specifically, we evaluate the performance of several GAI models, including GPT-4, in providing written corrective feedback compared to commercial AWCF tools and other advanced models. The study adopts a dual-method approach: (1) a comprehensive model evaluation using established English writing datasets to assess error correction performance via various prompting strategies, and (2) an empirical study involving teachers and students to examine the system's practical efficacy and users' perspective. Quantitative results indicate that GPT-4 with chain-of-thought prompting significantly outperforms commercial tools, achieving improved consistency and accuracy in error detection and correction. Qualitative feedback from participants further supports the system's potential to enhance students' writing quality and confidence, while concurrently reducing teachers' workload and optimizing instructional efficiency, despite concerns regarding occasional overcorrection. These findings emphasize the benefits of task-specific prompting in GAI-based AWCF systems and provide actionable insights for integrating advanced AI feedback into educational practices. |
|---|---|
| ISSN: | 0735-6331 1541-4140 |
| DOI: | 10.1177/07356331251359430 |
