Scoring Difficulty in Summary Writing Assessment: Toward the Reconstruction of Analytic Rubric
Saved in:
| Title: | Scoring Difficulty in Summary Writing Assessment: Toward the Reconstruction of Analytic Rubric |
|---|---|
| Language: | English |
| Authors: | Makiko Kato |
| Source: | Journal of Education and Learning. 2025 14(2):74-88. |
| Availability: | Canadian Center of Science and Education. 1595 Sixteenth Ave Suite 301, Richmond Hill, Ontario, L4B 3N9 Canada. Tel: 416-642-2606; Fax: 416-642-2608; e-mail: jel@ccsenet.org; Web site: http://www.ccsenet.org/journal/index.php/jel |
| Peer Reviewed: | Y |
| Page Count: | 15 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research Tests/Questionnaires |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Writing Evaluation, Scoring, Writing Skills, English (Second Language), Second Language Learning, College Students, Foreign Countries, Evaluators, Difficulty Level, Decision Making, Scoring Rubrics, College Faculty, Graduate Students, Student Attitudes, Teacher Attitudes |
| Geographic Terms: | Japan |
| ISSN: | 1927-5250 1927-5269 |
| Abstract: | This study aims to examine whether differences exist in the factors influencing the difficulty of scoring English summaries and determining scores based on the raters' attributes, and to collect candid opinions, considerations, and tentative suggestions for future improvements to the analytic rubric of summary writing for English learners. In this study, seven trained raters with diverse attribute backgrounds evaluated two kinds of English summaries written by Japanese university students using the analytic rubric with three evaluation items. A questionnaire was used to determine which of the three items were difficult to assess and why the raters perceived such difficulty, as well as what backgrounds and factors influenced their scoring decision-making. Moreover, through the raters' most recent experience, candid comments were collected for developing future rubrics. The results showed that whether the evaluators' attributes affected the difficulty of the evaluation was not clear. However, depending on the raters' experience in teaching English/assessing summary writing, requests for improvements in the descriptors of the evaluation items and in the rubric emerged. This study proposes a tentative analytic rubric for summary writing, providing a foundation for constructing a rubric that can be used more easily by future raters. It also highlights the opinions of expert and novice teachers conducting summary evaluations in education. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1469885 |
| Database: | ERIC |
| Abstract: | This study aims to examine whether differences exist in the factors influencing the difficulty of scoring English summaries and determining scores based on the raters' attributes, and to collect candid opinions, considerations, and tentative suggestions for future improvements to the analytic rubric of summary writing for English learners. In this study, seven trained raters with diverse attribute backgrounds evaluated two kinds of English summaries written by Japanese university students using the analytic rubric with three evaluation items. A questionnaire was used to determine which of the three items were difficult to assess and why the raters perceived such difficulty, as well as what backgrounds and factors influenced their scoring decision-making. Moreover, through the raters' most recent experience, candid comments were collected for developing future rubrics. The results showed that whether the evaluators' attributes affected the difficulty of the evaluation was not clear. However, depending on the raters' experience in teaching English/assessing summary writing, requests for improvements in the descriptors of the evaluation items and in the rubric emerged. This study proposes a tentative analytic rubric for summary writing, providing a foundation for constructing a rubric that can be used more easily by future raters. It also highlights the opinions of expert and novice teachers conducting summary evaluations in education. |
|---|---|
| ISSN: | 1927-5250 1927-5269 |
