View in EDS PDF Full Text

Issues of Question Equivalence in Online Exam Pools

Saved in:
Bibliographic Details
Title: Issues of Question Equivalence in Online Exam Pools
Language: English
Authors: Goolsby-Cole, Cody, Bass, Sarah M., Stanwyck, Liz, Leupen, Sarah, Carpenter, Tara S., Hodges, Linda C.
Source: Journal of College Science Teaching. Mar-Apr 2023 52(4):24-30.
Availability: National Science Teaching Association. 1840 Wilson Boulevard, Arlington, VA 22201-3000. Tel: 800-722-6782; Fax: 703-243-3924; e-mail: membership@nsta.org; Web site: https://www.nsta.org/
Peer Reviewed: Y
Page Count: 7
Publication Date: 2023
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Science Instruction, Computer Assisted Testing, Cheating, STEM Education, Test Items, Concept Formation, Scientific Concepts, Chemistry, Physics, Introductory Courses, Physiology, Science Achievement, Item Analysis, Science Tests, Biology, Majors (Students), Undergraduate Students, Learning Management Systems, Difficulty Level
ISSN: 0047-231X
1943-4898
Abstract: During the pandemic, the use of question pools for online testing was recommended to mitigate cheating, exposing multitudes of science, technology, engineering, and mathematics (STEM) students across the globe to this practice. Yet instructors may be unfamiliar with the ways that seemingly small changes between questions in a pool can expose differences in student understanding. In this study, we undertook an investigation of student performance on our questions in online exam pools across several STEM courses: upper-level physiology, general chemistry, and introductory physics. We found that the difficulty of creating analogous questions in a pool varied by question type, with quantitative problems being the easiest to vary without altering average student performance. However, when instructors created pools by rearranging aspects of a question, posing opposite counterparts of concepts, or formulating questions to assess the same learning objective, we sometimes discovered student learning differences between seemingly closely related ideas, illustrating the challenge of our own expert blind spot. We provide suggestions for how instructors can improve the equity of question pools, such as being cautious in how many variables one changes in a specific pool and "test driving" proposed questions in lower-stakes assessments.
Abstractor: As Provided
Entry Date: 2023
Access URL: https://www.nsta.org/journal-college-science-teaching/journal-college-science-teaching-marchapril-2023/issues-question
Accession Number: EJ1383886
Database: ERIC
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwE1E62_A6ZMc0SEjg6nGIP4AAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDHjJGgTnjM-iMQyjVgIBEICBmnIhkXc34OVkZahFMOQdiZfR-8yXrnPX89jIdt6hUAbAEbtjwY4s0ydXtpAEe8i5N5D1h0dUAITkepPTm-tCAsHNo0rv0qDlCdQgSz7bcSWiOxNcESV_LYaPQpyOxSGvScsCawwm9APK5t2jo5RsQYtW2sNk9HZllJiDQMQPCy63jjJ206Kcc-l1LRfUr-hszAa6f-YA1C9KZoE=
Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1383886
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Issues of Question Equivalence in Online Exam Pools
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Goolsby-Cole%2C+Cody%22">Goolsby-Cole, Cody</searchLink><br /><searchLink fieldCode="AR" term="%22Bass%2C+Sarah+M%2E%22">Bass, Sarah M.</searchLink><br /><searchLink fieldCode="AR" term="%22Stanwyck%2C+Liz%22">Stanwyck, Liz</searchLink><br /><searchLink fieldCode="AR" term="%22Leupen%2C+Sarah%22">Leupen, Sarah</searchLink><br /><searchLink fieldCode="AR" term="%22Carpenter%2C+Tara+S%2E%22">Carpenter, Tara S.</searchLink><br /><searchLink fieldCode="AR" term="%22Hodges%2C+Linda+C%2E%22">Hodges, Linda C.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+College+Science+Teaching%22"><i>Journal of College Science Teaching</i></searchLink>. Mar-Apr 2023 52(4):24-30.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: National Science Teaching Association. 1840 Wilson Boulevard, Arlington, VA 22201-3000. Tel: 800-722-6782; Fax: 703-243-3924; e-mail: membership@nsta.org; Web site: https://www.nsta.org/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 7
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Science+Instruction%22">Science Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Cheating%22">Cheating</searchLink><br /><searchLink fieldCode="DE" term="%22STEM+Education%22">STEM Education</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Concept+Formation%22">Concept Formation</searchLink><br /><searchLink fieldCode="DE" term="%22Scientific+Concepts%22">Scientific Concepts</searchLink><br /><searchLink fieldCode="DE" term="%22Chemistry%22">Chemistry</searchLink><br /><searchLink fieldCode="DE" term="%22Physics%22">Physics</searchLink><br /><searchLink fieldCode="DE" term="%22Introductory+Courses%22">Introductory Courses</searchLink><br /><searchLink fieldCode="DE" term="%22Physiology%22">Physiology</searchLink><br /><searchLink fieldCode="DE" term="%22Science+Achievement%22">Science Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Analysis%22">Item Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Science+Tests%22">Science Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Biology%22">Biology</searchLink><br /><searchLink fieldCode="DE" term="%22Majors+%28Students%29%22">Majors (Students)</searchLink><br /><searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Management+Systems%22">Learning Management Systems</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0047-231X<br />1943-4898
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: During the pandemic, the use of question pools for online testing was recommended to mitigate cheating, exposing multitudes of science, technology, engineering, and mathematics (STEM) students across the globe to this practice. Yet instructors may be unfamiliar with the ways that seemingly small changes between questions in a pool can expose differences in student understanding. In this study, we undertook an investigation of student performance on our questions in online exam pools across several STEM courses: upper-level physiology, general chemistry, and introductory physics. We found that the difficulty of creating analogous questions in a pool varied by question type, with quantitative problems being the easiest to vary without altering average student performance. However, when instructors created pools by rearranging aspects of a question, posing opposite counterparts of concepts, or formulating questions to assess the same learning objective, we sometimes discovered student learning differences between seemingly closely related ideas, illustrating the challenge of our own expert blind spot. We provide suggestions for how instructors can improve the equity of question pools, such as being cautious in how many variables one changes in a specific pool and "test driving" proposed questions in lower-stakes assessments.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: URL
  Label: Access URL
  Group: URL
  Data: <link linkTarget="URL" linkTerm="https://www.nsta.org/journal-college-science-teaching/journal-college-science-teaching-marchapril-2023/issues-question" linkWindow="_blank">https://www.nsta.org/journal-college-science-teaching/journal-college-science-teaching-marchapril-2023/issues-question</link>
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1383886
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1383886
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 7
        StartPage: 24
    Subjects:
      – SubjectFull: Science Instruction
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Cheating
        Type: general
      – SubjectFull: STEM Education
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Concept Formation
        Type: general
      – SubjectFull: Scientific Concepts
        Type: general
      – SubjectFull: Chemistry
        Type: general
      – SubjectFull: Physics
        Type: general
      – SubjectFull: Introductory Courses
        Type: general
      – SubjectFull: Physiology
        Type: general
      – SubjectFull: Science Achievement
        Type: general
      – SubjectFull: Item Analysis
        Type: general
      – SubjectFull: Science Tests
        Type: general
      – SubjectFull: Biology
        Type: general
      – SubjectFull: Majors (Students)
        Type: general
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: Learning Management Systems
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
    Titles:
      – TitleFull: Issues of Question Equivalence in Online Exam Pools
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Goolsby-Cole, Cody
      – PersonEntity:
          Name:
            NameFull: Bass, Sarah M.
      – PersonEntity:
          Name:
            NameFull: Stanwyck, Liz
      – PersonEntity:
          Name:
            NameFull: Leupen, Sarah
      – PersonEntity:
          Name:
            NameFull: Carpenter, Tara S.
      – PersonEntity:
          Name:
            NameFull: Hodges, Linda C.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2023
          Identifiers:
            – Type: issn-print
              Value: 0047-231X
            – Type: issn-electronic
              Value: 1943-4898
          Numbering:
            – Type: volume
              Value: 52
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Journal of College Science Teaching
              Type: main
ResultId 1