Validating a Claim-Evidence-Science Idea-Reasoning (CESR) Framework for Use in NGSS Assessment Tasks
Saved in:
| Title: | Validating a Claim-Evidence-Science Idea-Reasoning (CESR) Framework for Use in NGSS Assessment Tasks |
|---|---|
| Language: | English |
| Authors: | Hardcastle, Joseph M., Herrmann Abell, Cari F. (ORCID |
| Source: | Grantee Submission. 2021Paper presented at the NARST Virtual Annual Conference (Online, 2021). |
| Peer Reviewed: | Y |
| Page Count: | 12 |
| Publication Date: | 2021 |
| Sponsoring Agency: | Institute of Education Sciences (ED) |
| Contract Number: | R305A180512 |
| Document Type: | Reports - Research Speeches/Meeting Papers |
| Education Level: | Elementary Secondary Education |
| Descriptors: | Academic Standards, Energy, Scientific Concepts, Persuasive Discourse, Science Process Skills, Logical Thinking, Test Items, Scoring Rubrics, Validity, Reliability, Item Response Theory, Difficulty Level, Alignment (Education), Elementary Secondary Education |
| Abstract: | We developed assessment tasks aligned to the Next Generation Science Standards (NGSS) that require students to use argumentation and explanation practices along with disciplinary core ideas and crosscutting concepts to make sense of energy-related phenomena. Scoring rubrics were created to evaluate students' ability to make accurate claims, cite evidence, use relevant science ideas, and combine those elements to formulate well-reasoned arguments and explanations. We present an analysis of data to investigate the validity and reliability of our rubrics. Due to school closures caused by the COVID-19 pandemic, data were collected using Amazon Mechanical Turk (MTurk). The MTurk data were scored by two researchers to evaluate the inter-rater reliability. Data were then analyzed using Rasch modeling. Results show that rubric categories associated with stating claims, citing evidence, applying science ideas, and formulating coherent, well-reasoned arguments and explanations fit well to the Rasch model, and that rubric categories followed a hierarchy of difficulty. In this hierarchy, applying science ideas and formulating well-reasoned statements were more difficult than citing evidence, which were all more difficult than stating a claim. The ability to locate a student along this hierarchy allows for our tasks to be used to better understand a student's ability to write arguments and explanations of energy-related phenomena. |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2021 |
| Accession Number: | ED612227 |
| Database: | ERIC |
| Abstract: | We developed assessment tasks aligned to the Next Generation Science Standards (NGSS) that require students to use argumentation and explanation practices along with disciplinary core ideas and crosscutting concepts to make sense of energy-related phenomena. Scoring rubrics were created to evaluate students' ability to make accurate claims, cite evidence, use relevant science ideas, and combine those elements to formulate well-reasoned arguments and explanations. We present an analysis of data to investigate the validity and reliability of our rubrics. Due to school closures caused by the COVID-19 pandemic, data were collected using Amazon Mechanical Turk (MTurk). The MTurk data were scored by two researchers to evaluate the inter-rater reliability. Data were then analyzed using Rasch modeling. Results show that rubric categories associated with stating claims, citing evidence, applying science ideas, and formulating coherent, well-reasoned arguments and explanations fit well to the Rasch model, and that rubric categories followed a hierarchy of difficulty. In this hierarchy, applying science ideas and formulating well-reasoned statements were more difficult than citing evidence, which were all more difficult than stating a claim. The ability to locate a student along this hierarchy allows for our tasks to be used to better understand a student's ability to write arguments and explanations of energy-related phenomena. |
|---|
