View in EDS

From Sentence-Corrections to Deeper Dialogue: Qualitative Insights from LLM and Teacher Feedback on Student Writing. EdWorkingPaper No. 25-1193

Saved in:
Bibliographic Details
Title: From Sentence-Corrections to Deeper Dialogue: Qualitative Insights from LLM and Teacher Feedback on Student Writing. EdWorkingPaper No. 25-1193
Language: English
Authors: Christopher Mah, Mei Tan, Lena Phalen, Alexa Sparks, Dorottya Demszky, Annenberg Institute for School Reform at Brown University
Source: Annenberg Institute for School Reform at Brown University. 2025.
Availability: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
Peer Reviewed: N
Page Count: 33
Publication Date: 2025
Document Type: Reports - Research
Descriptors: Artificial Intelligence, Technology Uses in Education, Natural Language Processing, Writing Evaluation, Feedback (Response), Sentences, Error Correction, Revision (Written Composition), Writing Teachers, Writing Skills, Positive Reinforcement
Abstract: Effective writing feedback is a powerful tool for enhancing student learning, encouraging revision, and increasing motivation and agency. Yet, teachers face many challenges that prevent them from consistently providing effective writing feedback. Recent advances in generative artificial intelligence (AI) have led educators and researchers to experiment with AI tools powered by large language models (LLMs) to provide writing feedback, but research in this area has yielded mixed results. In this study, we used qualitative methods to compare LLM writing feedback and expert teacher (n = 12) feedback. Using a framework of dialogic writing feedback as our analytic lens, we highlight differences in LLM and teacher feedback along three dimensions: cognitive, social, and structural. We observed that LLMs primarily enacted corrective feedback at the sentence level and positioned students as novices requiring remediation. By contrast, we observed that teachers enacted more dialogic feedback, offering feedback at multiple levels and employing tactics that positioned students as agentic writers. Our findings support previous research describing limitations of LLM-based writing feedback. More importantly, our study contributes to the growing base of research by identifying specific feedback practices unique to highly skilled teachers that LLMs did not exhibit. These findings have implications for improving the quality of LLM feedback and shifting teachers' practice to foreground the types of writing feedback that best promote independent thinking and writing skills students will need in the age of generative AI. [Funding for this report was received from the Stanford Institute for Human-Centered AI and Stanford Accelerator for Learning.]
Abstractor: As Provided
Entry Date: 2025
Accession Number: ED674108
Database: ERIC
Description
Abstract:Effective writing feedback is a powerful tool for enhancing student learning, encouraging revision, and increasing motivation and agency. Yet, teachers face many challenges that prevent them from consistently providing effective writing feedback. Recent advances in generative artificial intelligence (AI) have led educators and researchers to experiment with AI tools powered by large language models (LLMs) to provide writing feedback, but research in this area has yielded mixed results. In this study, we used qualitative methods to compare LLM writing feedback and expert teacher (n = 12) feedback. Using a framework of dialogic writing feedback as our analytic lens, we highlight differences in LLM and teacher feedback along three dimensions: cognitive, social, and structural. We observed that LLMs primarily enacted corrective feedback at the sentence level and positioned students as novices requiring remediation. By contrast, we observed that teachers enacted more dialogic feedback, offering feedback at multiple levels and employing tactics that positioned students as agentic writers. Our findings support previous research describing limitations of LLM-based writing feedback. More importantly, our study contributes to the growing base of research by identifying specific feedback practices unique to highly skilled teachers that LLMs did not exhibit. These findings have implications for improving the quality of LLM feedback and shifting teachers' practice to foreground the types of writing feedback that best promote independent thinking and writing skills students will need in the age of generative AI. [Funding for this report was received from the Stanford Institute for Human-Centered AI and Stanford Accelerator for Learning.]