View in EDS

Scoring Difficulty in Summary Writing Assessment: Toward the Reconstruction of Analytic Rubric

Saved in:
Bibliographic Details
Title: Scoring Difficulty in Summary Writing Assessment: Toward the Reconstruction of Analytic Rubric
Language: English
Authors: Makiko Kato
Source: Journal of Education and Learning. 2025 14(2):74-88.
Availability: Canadian Center of Science and Education. 1595 Sixteenth Ave Suite 301, Richmond Hill, Ontario, L4B 3N9 Canada. Tel: 416-642-2606; Fax: 416-642-2608; e-mail: jel@ccsenet.org; Web site: http://www.ccsenet.org/journal/index.php/jel
Peer Reviewed: Y
Page Count: 15
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Tests/Questionnaires
Education Level: Higher Education
Postsecondary Education
Descriptors: Writing Evaluation, Scoring, Writing Skills, English (Second Language), Second Language Learning, College Students, Foreign Countries, Evaluators, Difficulty Level, Decision Making, Scoring Rubrics, College Faculty, Graduate Students, Student Attitudes, Teacher Attitudes
Geographic Terms: Japan
ISSN: 1927-5250
1927-5269
Abstract: This study aims to examine whether differences exist in the factors influencing the difficulty of scoring English summaries and determining scores based on the raters' attributes, and to collect candid opinions, considerations, and tentative suggestions for future improvements to the analytic rubric of summary writing for English learners. In this study, seven trained raters with diverse attribute backgrounds evaluated two kinds of English summaries written by Japanese university students using the analytic rubric with three evaluation items. A questionnaire was used to determine which of the three items were difficult to assess and why the raters perceived such difficulty, as well as what backgrounds and factors influenced their scoring decision-making. Moreover, through the raters' most recent experience, candid comments were collected for developing future rubrics. The results showed that whether the evaluators' attributes affected the difficulty of the evaluation was not clear. However, depending on the raters' experience in teaching English/assessing summary writing, requests for improvements in the descriptors of the evaluation items and in the rubric emerged. This study proposes a tentative analytic rubric for summary writing, providing a foundation for constructing a rubric that can be used more easily by future raters. It also highlights the opinions of expert and novice teachers conducting summary evaluations in education.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1469885
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1469885
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1469885
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Scoring Difficulty in Summary Writing Assessment: Toward the Reconstruction of Analytic Rubric
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Makiko+Kato%22">Makiko Kato</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Education+and+Learning%22"><i>Journal of Education and Learning</i></searchLink>. 2025 14(2):74-88.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Canadian Center of Science and Education. 1595 Sixteenth Ave Suite 301, Richmond Hill, Ontario, L4B 3N9 Canada. Tel: 416-642-2606; Fax: 416-642-2608; e-mail: jel@ccsenet.org; Web site: http://www.ccsenet.org/journal/index.php/jel
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 15
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research<br />Tests/Questionnaires
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Writing+Evaluation%22">Writing Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring%22">Scoring</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Skills%22">Writing Skills</searchLink><br /><searchLink fieldCode="DE" term="%22English+%28Second+Language%29%22">English (Second Language)</searchLink><br /><searchLink fieldCode="DE" term="%22Second+Language+Learning%22">Second Language Learning</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluators%22">Evaluators</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink><br /><searchLink fieldCode="DE" term="%22Decision+Making%22">Decision Making</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring+Rubrics%22">Scoring Rubrics</searchLink><br /><searchLink fieldCode="DE" term="%22College+Faculty%22">College Faculty</searchLink><br /><searchLink fieldCode="DE" term="%22Graduate+Students%22">Graduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Attitudes%22">Student Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Attitudes%22">Teacher Attitudes</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Japan%22">Japan</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1927-5250<br />1927-5269
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This study aims to examine whether differences exist in the factors influencing the difficulty of scoring English summaries and determining scores based on the raters' attributes, and to collect candid opinions, considerations, and tentative suggestions for future improvements to the analytic rubric of summary writing for English learners. In this study, seven trained raters with diverse attribute backgrounds evaluated two kinds of English summaries written by Japanese university students using the analytic rubric with three evaluation items. A questionnaire was used to determine which of the three items were difficult to assess and why the raters perceived such difficulty, as well as what backgrounds and factors influenced their scoring decision-making. Moreover, through the raters' most recent experience, candid comments were collected for developing future rubrics. The results showed that whether the evaluators' attributes affected the difficulty of the evaluation was not clear. However, depending on the raters' experience in teaching English/assessing summary writing, requests for improvements in the descriptors of the evaluation items and in the rubric emerged. This study proposes a tentative analytic rubric for summary writing, providing a foundation for constructing a rubric that can be used more easily by future raters. It also highlights the opinions of expert and novice teachers conducting summary evaluations in education.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1469885
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1469885
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 15
        StartPage: 74
    Subjects:
      – SubjectFull: Writing Evaluation
        Type: general
      – SubjectFull: Scoring
        Type: general
      – SubjectFull: Writing Skills
        Type: general
      – SubjectFull: English (Second Language)
        Type: general
      – SubjectFull: Second Language Learning
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Evaluators
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
      – SubjectFull: Decision Making
        Type: general
      – SubjectFull: Scoring Rubrics
        Type: general
      – SubjectFull: College Faculty
        Type: general
      – SubjectFull: Graduate Students
        Type: general
      – SubjectFull: Student Attitudes
        Type: general
      – SubjectFull: Teacher Attitudes
        Type: general
      – SubjectFull: Japan
        Type: general
    Titles:
      – TitleFull: Scoring Difficulty in Summary Writing Assessment: Toward the Reconstruction of Analytic Rubric
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Makiko Kato
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 1927-5250
            – Type: issn-electronic
              Value: 1927-5269
          Numbering:
            – Type: volume
              Value: 14
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Journal of Education and Learning
              Type: main
ResultId 1