Assessing student perceptions and use of instructor versus AI‐generated feedback.
Saved in:
| Title: | Assessing student perceptions and use of instructor versus AI‐generated feedback. |
|---|---|
| Authors: | Er, Erkan1 (AUTHOR) erkane@metu.edu.tr, Akçapınar, Gökhan2 (AUTHOR), Bayazıt, Alper3 (AUTHOR), Noroozi, Omid4 (AUTHOR), Banihashem, Seyyed Kazem4,5 (AUTHOR) |
| Source: | British Journal of Educational Technology. May2025, Vol. 56 Issue 3, p1074-1091. 18p. |
| Subject Terms: | *Psychology of students, *Artificial intelligence, *Psychological feedback, Language models, Statistical significance, ChatGPT |
| Abstract: | Despite the growing research interest in the use of large language models for feedback provision, it still remains unknown how students perceive and use AI‐generated feedback compared to instructor feedback in authentic settings. To address this gap, this study compared instructor and AI‐generated feedback in a Java programming course through an experimental research design where students were randomly assigned to either condition. Both feedback providers used the same assessment rubric, and students were asked to improve their work based on the feedback. The feedback perceptions scale and students' laboratory assignment scores were compared in both conditions. Results showed that students perceived instructor feedback as significantly more useful than AI feedback. While instructor feedback was also perceived as more fair, developmental and encouraging, these differences were not statistically significant. Importantly, students receiving instructor feedback showed significantly greater improvements in their lab scores compared to those receiving AI feedback, even after controlling for their initial knowledge levels. Based on the findings, we posit that AI models potentially need to be trained on data specific to educational contexts and hybrid feedback models that combine AI's and instructors' strengths should be considered for effective feedback practices. Practitioner notesWhat is already known about this topicFeedback is crucial for student learning in programming education.Providing detailed personalised feedback is challenging for instructors.AI‐powered solutions like ChatGPT can be effective in feedback provision.Existing research is limited and shows mixed results about AI‐generated feedback.What this paper addsThe effectiveness of AI‐generated feedback was compared to instructor feedback.Both feedback types received positive perceptions, but instructor feedback was seen as more useful.Instructor feedback led to greater score improvements in the programming task.Implications for practice and/or policyAI should not be the sole source of feedback, as human expertise is crucial.AI models should be trained on context‐specific data to improve feedback actionability.Hybrid feedback models should be considered for a scalable and effective approach. [ABSTRACT FROM AUTHOR] |
| Copyright of British Journal of Educational Technology is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 184338620 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Assessing student perceptions and use of instructor versus AI‐generated feedback. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Er%2C+Erkan%22">Er, Erkan</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> erkane@metu.edu.tr</i><br /><searchLink fieldCode="AR" term="%22Akçapınar%2C+Gökhan%22">Akçapınar, Gökhan</searchLink><relatesTo>2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Bayazıt%2C+Alper%22">Bayazıt, Alper</searchLink><relatesTo>3</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Noroozi%2C+Omid%22">Noroozi, Omid</searchLink><relatesTo>4</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Banihashem%2C+Seyyed+Kazem%22">Banihashem, Seyyed Kazem</searchLink><relatesTo>4,5</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22British+Journal+of+Educational+Technology%22">British Journal of Educational Technology</searchLink>. May2025, Vol. 56 Issue 3, p1074-1091. 18p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Psychology+of+students%22">Psychology of students</searchLink><br />*<searchLink fieldCode="DE" term="%22Artificial+intelligence%22">Artificial intelligence</searchLink><br />*<searchLink fieldCode="DE" term="%22Psychological+feedback%22">Psychological feedback</searchLink><br /><searchLink fieldCode="DE" term="%22Language+models%22">Language models</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+significance%22">Statistical significance</searchLink><br /><searchLink fieldCode="DE" term="%22ChatGPT%22">ChatGPT</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Despite the growing research interest in the use of large language models for feedback provision, it still remains unknown how students perceive and use AI‐generated feedback compared to instructor feedback in authentic settings. To address this gap, this study compared instructor and AI‐generated feedback in a Java programming course through an experimental research design where students were randomly assigned to either condition. Both feedback providers used the same assessment rubric, and students were asked to improve their work based on the feedback. The feedback perceptions scale and students' laboratory assignment scores were compared in both conditions. Results showed that students perceived instructor feedback as significantly more useful than AI feedback. While instructor feedback was also perceived as more fair, developmental and encouraging, these differences were not statistically significant. Importantly, students receiving instructor feedback showed significantly greater improvements in their lab scores compared to those receiving AI feedback, even after controlling for their initial knowledge levels. Based on the findings, we posit that AI models potentially need to be trained on data specific to educational contexts and hybrid feedback models that combine AI's and instructors' strengths should be considered for effective feedback practices. Practitioner notesWhat is already known about this topicFeedback is crucial for student learning in programming education.Providing detailed personalised feedback is challenging for instructors.AI‐powered solutions like ChatGPT can be effective in feedback provision.Existing research is limited and shows mixed results about AI‐generated feedback.What this paper addsThe effectiveness of AI‐generated feedback was compared to instructor feedback.Both feedback types received positive perceptions, but instructor feedback was seen as more useful.Instructor feedback led to greater score improvements in the programming task.Implications for practice and/or policyAI should not be the sole source of feedback, as human expertise is crucial.AI models should be trained on context‐specific data to improve feedback actionability.Hybrid feedback models should be considered for a scalable and effective approach. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of British Journal of Educational Technology is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=184338620 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1111/bjet.13558 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 18 StartPage: 1074 Subjects: – SubjectFull: Psychology of students Type: general – SubjectFull: Artificial intelligence Type: general – SubjectFull: Psychological feedback Type: general – SubjectFull: Language models Type: general – SubjectFull: Statistical significance Type: general – SubjectFull: ChatGPT Type: general Titles: – TitleFull: Assessing student perceptions and use of instructor versus AI‐generated feedback. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Er, Erkan – PersonEntity: Name: NameFull: Akçapınar, Gökhan – PersonEntity: Name: NameFull: Bayazıt, Alper – PersonEntity: Name: NameFull: Noroozi, Omid – PersonEntity: Name: NameFull: Banihashem, Seyyed Kazem IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 05 Text: May2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 00071013 Numbering: – Type: volume Value: 56 – Type: issue Value: 3 Titles: – TitleFull: British Journal of Educational Technology Type: main |
| ResultId | 1 |
