Evaluation of the Impact of Equating Approach on the Parameter and Student Score Stability Using Pre- and Post-Equated Designs in the Post-Pandemic Environment
Saved in:
| Title: | Evaluation of the Impact of Equating Approach on the Parameter and Student Score Stability Using Pre- and Post-Equated Designs in the Post-Pandemic Environment |
|---|---|
| Language: | English |
| Authors: | Tomkowicz, Joanna, Kim, Dong-In, Wan, Ping |
| Source: | Online Submission. 2022Paper presented at the Annual Meeting of the National Council on Measurement in Education (San Diego, CA, Apr 2022). |
| Peer Reviewed: | N |
| Page Count: | 38 |
| Publication Date: | 2022 |
| Document Type: | Speeches/Meeting Papers Reports - Research Numerical/Quantitative Data |
| Education Level: | Elementary Education Junior High Schools Middle Schools Secondary Education |
| Descriptors: | Equated Scores, Test Items, Evaluation Methods, Pandemics, English Instruction, Language Arts, Mathematics Education, Elementary School Students, Middle School Students, Multiple Choice Tests, Student Evaluation, Scoring, Language Tests, Mathematics Tests |
| Abstract: | In this study we evaluated the stability of item parameters and student scores, using the pre-equated (pre-pandemic) parameters from Spring 2019 and post-equated (post-pandemic) parameters from Spring 2021 in two calibration and equating designs related to item parameter treatment: re-estimating all anchor parameters (Design 1) and holding the c-parameter fixed for all multiple-choice items (Design 2). The Spring 2021 English Language Arts and Mathematics grades 3 through 8 operational test data from a large-scale testing program were used for this study. It was found that re-estimating item parameters in both Design 1 and Design 2 had little effect on the calibration and equating results, indicating acceptable stability of item parameters re-estimated in the post-pandemic environment. There was little or no difference in the mean scale scores and percentages of students classified in the different performance levels when students were scored using the pre-equated versus post-equated item parameters in Spring 2021. These differences in performance were even smaller when scores obtained in post-equated Design 1 and post-equated Design 2 were compared. When Spring 2021 student performance was compared to the Spring 2019 student performance, a decrease in performance was observed in each grade and content area that could be attributed to the effect of the pandemic and disrupted learning in the 2020-21 school year. |
| Abstractor: | As Provided |
| Entry Date: | 2023 |
| Accession Number: | ED624578 |
| Database: | ERIC |
| Abstract: | In this study we evaluated the stability of item parameters and student scores, using the pre-equated (pre-pandemic) parameters from Spring 2019 and post-equated (post-pandemic) parameters from Spring 2021 in two calibration and equating designs related to item parameter treatment: re-estimating all anchor parameters (Design 1) and holding the c-parameter fixed for all multiple-choice items (Design 2). The Spring 2021 English Language Arts and Mathematics grades 3 through 8 operational test data from a large-scale testing program were used for this study. It was found that re-estimating item parameters in both Design 1 and Design 2 had little effect on the calibration and equating results, indicating acceptable stability of item parameters re-estimated in the post-pandemic environment. There was little or no difference in the mean scale scores and percentages of students classified in the different performance levels when students were scored using the pre-equated versus post-equated item parameters in Spring 2021. These differences in performance were even smaller when scores obtained in post-equated Design 1 and post-equated Design 2 were compared. When Spring 2021 student performance was compared to the Spring 2019 student performance, a decrease in performance was observed in each grade and content area that could be attributed to the effect of the pandemic and disrupted learning in the 2020-21 school year. |
|---|
