View in EDS PDF Full Text

Using Topic Models to Understand Rater-Mediated Writing Assessments

Saved in:
Bibliographic Details
Title: Using Topic Models to Understand Rater-Mediated Writing Assessments
Language: English
Authors: Jordan M. Wheeler (ORCID 0000-0001-9766-5014), Kevin R. Raczynski, Allan S. Cohen (ORCID 0000-0002-8776-9378), George Engelhard (ORCID 0000-0002-1694-8942)
Source: Journal of Experimental Education. 2024 92(1):186-205.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 20
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Education Level: Junior High Schools
Middle Schools
Secondary Education
Elementary Education
Grade 6
Intermediate Grades
Grade 8
Descriptors: Middle School Students, Grade 6, Grade 8, Mathematical Models, Word Processing, Cognitive Mapping, Writing Strategies, Performance Technology
DOI: 10.1080/00220973.2022.2130129
ISSN: 0022-0973
1940-0683
Abstract: Integrated writing assessments are tasks that require students to construct an essay using evidence from source texts. Topic models are statistical models used to analyze text. This study analyzes two integrated writing tasks from students in grades 6 (N=1,250) and 8 (N=1,250) using topic models. Independent evaluations of the essays were also conducted to help generalize the results across the two grades and better understand students' underlying writing techniques. Four main categories were identified: (1) use of words that appeared in the prompt, (2) words that appeared in the source texts, (3) words that reflected inference, and (4) words that reflected integration. Higher-scoring students used words that reflect inference and integration of source text more often than lower-performing students.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1404995
Database: ERIC
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwF8m9k0Z6xqcpAsyMmTUwwIAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDFzosvkHRSAiHgvGcQIBEICBmzx9uu_YWMWp755eWkMs5fRT3yG2sdjpinf_hljPZwZbSBkOV6_Otxj7PcXv4E-0ouaKKzeni7rZ8RAnyjpVjUBGRz1q6aGVKfueF0jeyaaf5KwJ0XjboHxuAt3bj0rB4GAVv8hr9Db2DXBQcDdeRUhq9RpTUX-1U_hRuBhgVbB_Ogd3CsLZLHYD1r89tWqb4c2qLulYU8FqMRx3
Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1404995
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Using Topic Models to Understand Rater-Mediated Writing Assessments
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Jordan+M%2E+Wheeler%22">Jordan M. Wheeler</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0001-9766-5014">0000-0001-9766-5014</externalLink>)<br /><searchLink fieldCode="AR" term="%22Kevin+R%2E+Raczynski%22">Kevin R. Raczynski</searchLink><br /><searchLink fieldCode="AR" term="%22Allan+S%2E+Cohen%22">Allan S. Cohen</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-8776-9378">0000-0002-8776-9378</externalLink>)<br /><searchLink fieldCode="AR" term="%22George+Engelhard%22">George Engelhard</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-1694-8942">0000-0002-1694-8942</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Experimental+Education%22"><i>Journal of Experimental Education</i></searchLink>. 2024 92(1):186-205.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 20
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Junior+High+Schools%22">Junior High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+6%22">Grade 6</searchLink><br /><searchLink fieldCode="EL" term="%22Intermediate+Grades%22">Intermediate Grades</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+8%22">Grade 8</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Middle+School+Students%22">Middle School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Grade+6%22">Grade 6</searchLink><br /><searchLink fieldCode="DE" term="%22Grade+8%22">Grade 8</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematical+Models%22">Mathematical Models</searchLink><br /><searchLink fieldCode="DE" term="%22Word+Processing%22">Word Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Cognitive+Mapping%22">Cognitive Mapping</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Strategies%22">Writing Strategies</searchLink><br /><searchLink fieldCode="DE" term="%22Performance+Technology%22">Performance Technology</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/00220973.2022.2130129
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0022-0973<br />1940-0683
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Integrated writing assessments are tasks that require students to construct an essay using evidence from source texts. Topic models are statistical models used to analyze text. This study analyzes two integrated writing tasks from students in grades 6 (N=1,250) and 8 (N=1,250) using topic models. Independent evaluations of the essays were also conducted to help generalize the results across the two grades and better understand students' underlying writing techniques. Four main categories were identified: (1) use of words that appeared in the prompt, (2) words that appeared in the source texts, (3) words that reflected inference, and (4) words that reflected integration. Higher-scoring students used words that reflect inference and integration of source text more often than lower-performing students.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1404995
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1404995
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/00220973.2022.2130129
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 20
        StartPage: 186
    Subjects:
      – SubjectFull: Middle School Students
        Type: general
      – SubjectFull: Grade 6
        Type: general
      – SubjectFull: Grade 8
        Type: general
      – SubjectFull: Mathematical Models
        Type: general
      – SubjectFull: Word Processing
        Type: general
      – SubjectFull: Cognitive Mapping
        Type: general
      – SubjectFull: Writing Strategies
        Type: general
      – SubjectFull: Performance Technology
        Type: general
    Titles:
      – TitleFull: Using Topic Models to Understand Rater-Mediated Writing Assessments
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Jordan M. Wheeler
      – PersonEntity:
          Name:
            NameFull: Kevin R. Raczynski
      – PersonEntity:
          Name:
            NameFull: Allan S. Cohen
      – PersonEntity:
          Name:
            NameFull: George Engelhard
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 0022-0973
            – Type: issn-electronic
              Value: 1940-0683
          Numbering:
            – Type: volume
              Value: 92
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Journal of Experimental Education
              Type: main
ResultId 1