View in EDS

Ensuring Breadth and Depth of Knowledge on Multiple-Choice Examinations for Board Certification

Saved in:
Bibliographic Details
Title: Ensuring Breadth and Depth of Knowledge on Multiple-Choice Examinations for Board Certification
Language: English
Authors: Heath Kincaid (ORCID 0009-0006-3759-4086), Anthony Moreno-Sparks (ORCID 0000-0002-0277-497X), Pooja Shivraj (ORCID 0000-0002-4158-3084), Jill Holmes, Amy Young (ORCID 0000-0003-0447-699X), George D. Wendel
Source: Practical Assessment, Research & Evaluation. 2025 30(1).
Availability: University of Massachusetts Amherst Libraries. 154 Hicks Way, Amherst, MA 01003. e-mail: pare@umass.edu; Web site: https://openpublishing.library.umass.edu/pare/
Peer Reviewed: Y
Page Count: 16
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Descriptors: Multiple Choice Tests, Certification, Natural Language Processing, Gynecology, Obstetrics, Physicians, Algorithms, Test Content, Content Analysis
ISSN: 1531-7714
Abstract: Certification organizations aim to assess candidates on their breadth and depth of knowledge to determine eligibility for certification in their field of specialty. Assessments used for certification, when appropriately constructed, should use questions (or items) that assess the entirety of the field. However, comparing the plethora of the content of items to assess content coverage is a lengthy and time-consuming process. In an effort to become more aligned with the purpose of increasing content representativeness, organizations can implement a variety of Natural Language Processing (NLP) techniques with their items to ensure no one concept, medical condition, or scenario presents itself redundantly throughout each of its multiple-choice examinations. We provide an illustrative example from the American Board of Obstetrics and Gynecology (ABOG) of the NLP processes used to increase efficiencies and ensure content representativeness.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1491707
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1491707
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1491707
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Ensuring Breadth and Depth of Knowledge on Multiple-Choice Examinations for Board Certification
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Heath+Kincaid%22">Heath Kincaid</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0006-3759-4086">0009-0006-3759-4086</externalLink>)<br /><searchLink fieldCode="AR" term="%22Anthony+Moreno-Sparks%22">Anthony Moreno-Sparks</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-0277-497X">0000-0002-0277-497X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Pooja+Shivraj%22">Pooja Shivraj</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-4158-3084">0000-0002-4158-3084</externalLink>)<br /><searchLink fieldCode="AR" term="%22Jill+Holmes%22">Jill Holmes</searchLink><br /><searchLink fieldCode="AR" term="%22Amy+Young%22">Amy Young</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-0447-699X">0000-0003-0447-699X</externalLink>)<br /><searchLink fieldCode="AR" term="%22George+D%2E+Wendel%22">George D. Wendel</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Practical+Assessment%2C+Research+%26+Evaluation%22"><i>Practical Assessment, Research & Evaluation</i></searchLink>. 2025 30(1).
– Name: Avail
  Label: Availability
  Group: Avail
  Data: University of Massachusetts Amherst Libraries. 154 Hicks Way, Amherst, MA 01003. e-mail: pare@umass.edu; Web site: https://openpublishing.library.umass.edu/pare/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 16
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Multiple+Choice+Tests%22">Multiple Choice Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Certification%22">Certification</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Gynecology%22">Gynecology</searchLink><br /><searchLink fieldCode="DE" term="%22Obstetrics%22">Obstetrics</searchLink><br /><searchLink fieldCode="DE" term="%22Physicians%22">Physicians</searchLink><br /><searchLink fieldCode="DE" term="%22Algorithms%22">Algorithms</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Content%22">Test Content</searchLink><br /><searchLink fieldCode="DE" term="%22Content+Analysis%22">Content Analysis</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1531-7714
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Certification organizations aim to assess candidates on their breadth and depth of knowledge to determine eligibility for certification in their field of specialty. Assessments used for certification, when appropriately constructed, should use questions (or items) that assess the entirety of the field. However, comparing the plethora of the content of items to assess content coverage is a lengthy and time-consuming process. In an effort to become more aligned with the purpose of increasing content representativeness, organizations can implement a variety of Natural Language Processing (NLP) techniques with their items to ensure no one concept, medical condition, or scenario presents itself redundantly throughout each of its multiple-choice examinations. We provide an illustrative example from the American Board of Obstetrics and Gynecology (ABOG) of the NLP processes used to increase efficiencies and ensure content representativeness.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1491707
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1491707
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 16
    Subjects:
      – SubjectFull: Multiple Choice Tests
        Type: general
      – SubjectFull: Certification
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Gynecology
        Type: general
      – SubjectFull: Obstetrics
        Type: general
      – SubjectFull: Physicians
        Type: general
      – SubjectFull: Algorithms
        Type: general
      – SubjectFull: Test Content
        Type: general
      – SubjectFull: Content Analysis
        Type: general
    Titles:
      – TitleFull: Ensuring Breadth and Depth of Knowledge on Multiple-Choice Examinations for Board Certification
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Heath Kincaid
      – PersonEntity:
          Name:
            NameFull: Anthony Moreno-Sparks
      – PersonEntity:
          Name:
            NameFull: Pooja Shivraj
      – PersonEntity:
          Name:
            NameFull: Jill Holmes
      – PersonEntity:
          Name:
            NameFull: Amy Young
      – PersonEntity:
          Name:
            NameFull: George D. Wendel
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-electronic
              Value: 1531-7714
          Numbering:
            – Type: volume
              Value: 30
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Practical Assessment, Research & Evaluation
              Type: main
ResultId 1