View in EDS

Evaluation of the Impact of Equating Approach on the Parameter and Student Score Stability Using Pre- and Post-Equated Designs in the Post-Pandemic Environment

Saved in:
Bibliographic Details
Title: Evaluation of the Impact of Equating Approach on the Parameter and Student Score Stability Using Pre- and Post-Equated Designs in the Post-Pandemic Environment
Language: English
Authors: Tomkowicz, Joanna, Kim, Dong-In, Wan, Ping
Source: Online Submission. 2022Paper presented at the Annual Meeting of the National Council on Measurement in Education (San Diego, CA, Apr 2022).
Peer Reviewed: N
Page Count: 38
Publication Date: 2022
Document Type: Speeches/Meeting Papers
Reports - Research
Numerical/Quantitative Data
Education Level: Elementary Education
Junior High Schools
Middle Schools
Secondary Education
Descriptors: Equated Scores, Test Items, Evaluation Methods, Pandemics, English Instruction, Language Arts, Mathematics Education, Elementary School Students, Middle School Students, Multiple Choice Tests, Student Evaluation, Scoring, Language Tests, Mathematics Tests
Abstract: In this study we evaluated the stability of item parameters and student scores, using the pre-equated (pre-pandemic) parameters from Spring 2019 and post-equated (post-pandemic) parameters from Spring 2021 in two calibration and equating designs related to item parameter treatment: re-estimating all anchor parameters (Design 1) and holding the c-parameter fixed for all multiple-choice items (Design 2). The Spring 2021 English Language Arts and Mathematics grades 3 through 8 operational test data from a large-scale testing program were used for this study. It was found that re-estimating item parameters in both Design 1 and Design 2 had little effect on the calibration and equating results, indicating acceptable stability of item parameters re-estimated in the post-pandemic environment. There was little or no difference in the mean scale scores and percentages of students classified in the different performance levels when students were scored using the pre-equated versus post-equated item parameters in Spring 2021. These differences in performance were even smaller when scores obtained in post-equated Design 1 and post-equated Design 2 were compared. When Spring 2021 student performance was compared to the Spring 2019 student performance, a decrease in performance was observed in each grade and content area that could be attributed to the effect of the pandemic and disrupted learning in the 2020-21 school year.
Abstractor: As Provided
Entry Date: 2023
Accession Number: ED624578
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED624578
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED624578
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Evaluation of the Impact of Equating Approach on the Parameter and Student Score Stability Using Pre- and Post-Equated Designs in the Post-Pandemic Environment
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Tomkowicz%2C+Joanna%22">Tomkowicz, Joanna</searchLink><br /><searchLink fieldCode="AR" term="%22Kim%2C+Dong-In%22">Kim, Dong-In</searchLink><br /><searchLink fieldCode="AR" term="%22Wan%2C+Ping%22">Wan, Ping</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Online+Submission%22"><i>Online Submission</i></searchLink>. 2022Paper presented at the Annual Meeting of the National Council on Measurement in Education (San Diego, CA, Apr 2022).
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 38
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research<br />Numerical/Quantitative Data
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Junior+High+Schools%22">Junior High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Equated+Scores%22">Equated Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Pandemics%22">Pandemics</searchLink><br /><searchLink fieldCode="DE" term="%22English+Instruction%22">English Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Arts%22">Language Arts</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Education%22">Mathematics Education</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+School+Students%22">Elementary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Middle+School+Students%22">Middle School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Multiple+Choice+Tests%22">Multiple Choice Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Evaluation%22">Student Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring%22">Scoring</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Tests%22">Language Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Tests%22">Mathematics Tests</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In this study we evaluated the stability of item parameters and student scores, using the pre-equated (pre-pandemic) parameters from Spring 2019 and post-equated (post-pandemic) parameters from Spring 2021 in two calibration and equating designs related to item parameter treatment: re-estimating all anchor parameters (Design 1) and holding the c-parameter fixed for all multiple-choice items (Design 2). The Spring 2021 English Language Arts and Mathematics grades 3 through 8 operational test data from a large-scale testing program were used for this study. It was found that re-estimating item parameters in both Design 1 and Design 2 had little effect on the calibration and equating results, indicating acceptable stability of item parameters re-estimated in the post-pandemic environment. There was little or no difference in the mean scale scores and percentages of students classified in the different performance levels when students were scored using the pre-equated versus post-equated item parameters in Spring 2021. These differences in performance were even smaller when scores obtained in post-equated Design 1 and post-equated Design 2 were compared. When Spring 2021 student performance was compared to the Spring 2019 student performance, a decrease in performance was observed in each grade and content area that could be attributed to the effect of the pandemic and disrupted learning in the 2020-21 school year.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED624578
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED624578
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 38
    Subjects:
      – SubjectFull: Equated Scores
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Evaluation Methods
        Type: general
      – SubjectFull: Pandemics
        Type: general
      – SubjectFull: English Instruction
        Type: general
      – SubjectFull: Language Arts
        Type: general
      – SubjectFull: Mathematics Education
        Type: general
      – SubjectFull: Elementary School Students
        Type: general
      – SubjectFull: Middle School Students
        Type: general
      – SubjectFull: Multiple Choice Tests
        Type: general
      – SubjectFull: Student Evaluation
        Type: general
      – SubjectFull: Scoring
        Type: general
      – SubjectFull: Language Tests
        Type: general
      – SubjectFull: Mathematics Tests
        Type: general
    Titles:
      – TitleFull: Evaluation of the Impact of Equating Approach on the Parameter and Student Score Stability Using Pre- and Post-Equated Designs in the Post-Pandemic Environment
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Tomkowicz, Joanna
      – PersonEntity:
          Name:
            NameFull: Kim, Dong-In
      – PersonEntity:
          Name:
            NameFull: Wan, Ping
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 23
              M: 04
              Type: published
              Y: 2022
          Titles:
            – TitleFull: Online Submission
              Type: main
ResultId 1