AI-Assisted Automated Scoring of Picture-Cued Writing Tasks for Language Assessment
Saved in:
| Title: | AI-Assisted Automated Scoring of Picture-Cued Writing Tasks for Language Assessment |
|---|---|
| Language: | English |
| Authors: | Zhao, Ruibin, Zhuang, Yipeng, Zou, Di, Xie, Qin, Yu, Philip L. H. |
| Source: | Education and Information Technologies. Jun 2023 28(6):7031-7063. |
| Availability: | Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/ |
| Peer Reviewed: | Y |
| Page Count: | 33 |
| Publication Date: | 2023 |
| Document Type: | Journal Articles Reports - Research Tests/Questionnaires |
| Education Level: | Elementary Education Secondary Education |
| Descriptors: | Artificial Intelligence, Automation, Scoring, Visual Aids, Writing (Composition), Language Tests, Elementary School Students, Secondary School Students |
| DOI: | 10.1007/s10639-022-11473-y |
| ISSN: | 1360-2357 1573-7608 |
| Abstract: | Grading assignments is inherently subjective and time-consuming; automatic scoring tools can greatly reduce teacher workload and shorten the time needed for providing feedback to learners. The purpose of this paper is to propose a novel method for automatically scoring student responses to picture-cued writing tasks. As a popular paradigm for language instruction and assessment, a picture-cued writing task typically requires students to describe a picture or pictures. Correspondingly, the automatic scoring methods must measure the link(s) between visual pictures and their textual descriptions. For this purpose, we first designed a picture-cued writing test and collected nearly 4 k responses from 279 K12 students. Based on these responses, we then developed an AI scoring model by incorporating the emerging cross-modal matching technology and some NLP algorithms. The performance of the model was evaluated carefully with six popular measures and was found to demonstrate accurate scoring results with a small mean absolute error of 0.479 and a high adjacent-agreement rate of 90.64%. We believe this method could reduce the subjective elements inherent in human grading and save teachers' time from the mundane task of grading to other valuable endeavors such as designing teaching plans based on AI-generated diagnosis of student progress. |
| Abstractor: | As Provided |
| Entry Date: | 2023 |
| Accession Number: | EJ1378486 |
| Database: | ERIC |
| Abstract: | Grading assignments is inherently subjective and time-consuming; automatic scoring tools can greatly reduce teacher workload and shorten the time needed for providing feedback to learners. The purpose of this paper is to propose a novel method for automatically scoring student responses to picture-cued writing tasks. As a popular paradigm for language instruction and assessment, a picture-cued writing task typically requires students to describe a picture or pictures. Correspondingly, the automatic scoring methods must measure the link(s) between visual pictures and their textual descriptions. For this purpose, we first designed a picture-cued writing test and collected nearly 4 k responses from 279 K12 students. Based on these responses, we then developed an AI scoring model by incorporating the emerging cross-modal matching technology and some NLP algorithms. The performance of the model was evaluated carefully with six popular measures and was found to demonstrate accurate scoring results with a small mean absolute error of 0.479 and a high adjacent-agreement rate of 90.64%. We believe this method could reduce the subjective elements inherent in human grading and save teachers' time from the mundane task of grading to other valuable endeavors such as designing teaching plans based on AI-generated diagnosis of student progress. |
|---|---|
| ISSN: | 1360-2357 1573-7608 |
| DOI: | 10.1007/s10639-022-11473-y |
