Issues of Question Equivalence in Online Exam Pools
Saved in:
| Title: | Issues of Question Equivalence in Online Exam Pools |
|---|---|
| Language: | English |
| Authors: | Goolsby-Cole, Cody, Bass, Sarah M., Stanwyck, Liz, Leupen, Sarah, Carpenter, Tara S., Hodges, Linda C. |
| Source: | Journal of College Science Teaching. Mar-Apr 2023 52(4):24-30. |
| Availability: | National Science Teaching Association. 1840 Wilson Boulevard, Arlington, VA 22201-3000. Tel: 800-722-6782; Fax: 703-243-3924; e-mail: membership@nsta.org; Web site: https://www.nsta.org/ |
| Peer Reviewed: | Y |
| Page Count: | 7 |
| Publication Date: | 2023 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Science Instruction, Computer Assisted Testing, Cheating, STEM Education, Test Items, Concept Formation, Scientific Concepts, Chemistry, Physics, Introductory Courses, Physiology, Science Achievement, Item Analysis, Science Tests, Biology, Majors (Students), Undergraduate Students, Learning Management Systems, Difficulty Level |
| ISSN: | 0047-231X 1943-4898 |
| Abstract: | During the pandemic, the use of question pools for online testing was recommended to mitigate cheating, exposing multitudes of science, technology, engineering, and mathematics (STEM) students across the globe to this practice. Yet instructors may be unfamiliar with the ways that seemingly small changes between questions in a pool can expose differences in student understanding. In this study, we undertook an investigation of student performance on our questions in online exam pools across several STEM courses: upper-level physiology, general chemistry, and introductory physics. We found that the difficulty of creating analogous questions in a pool varied by question type, with quantitative problems being the easiest to vary without altering average student performance. However, when instructors created pools by rearranging aspects of a question, posing opposite counterparts of concepts, or formulating questions to assess the same learning objective, we sometimes discovered student learning differences between seemingly closely related ideas, illustrating the challenge of our own expert blind spot. We provide suggestions for how instructors can improve the equity of question pools, such as being cautious in how many variables one changes in a specific pool and "test driving" proposed questions in lower-stakes assessments. |
| Abstractor: | As Provided |
| Entry Date: | 2023 |
| Access URL: | https://www.nsta.org/journal-college-science-teaching/journal-college-science-teaching-marchapril-2023/issues-question |
| Accession Number: | EJ1383886 |
| Database: | ERIC |
| Abstract: | During the pandemic, the use of question pools for online testing was recommended to mitigate cheating, exposing multitudes of science, technology, engineering, and mathematics (STEM) students across the globe to this practice. Yet instructors may be unfamiliar with the ways that seemingly small changes between questions in a pool can expose differences in student understanding. In this study, we undertook an investigation of student performance on our questions in online exam pools across several STEM courses: upper-level physiology, general chemistry, and introductory physics. We found that the difficulty of creating analogous questions in a pool varied by question type, with quantitative problems being the easiest to vary without altering average student performance. However, when instructors created pools by rearranging aspects of a question, posing opposite counterparts of concepts, or formulating questions to assess the same learning objective, we sometimes discovered student learning differences between seemingly closely related ideas, illustrating the challenge of our own expert blind spot. We provide suggestions for how instructors can improve the equity of question pools, such as being cautious in how many variables one changes in a specific pool and "test driving" proposed questions in lower-stakes assessments. |
|---|---|
| ISSN: | 0047-231X 1943-4898 |
