View in EDS

Classification of Open-Ended Responses to a Research-Based Assessment Using Natural Language Processing

Saved in:
Bibliographic Details
Title: Classification of Open-Ended Responses to a Research-Based Assessment Using Natural Language Processing
Language: English
Authors: Wilson, Joseph (ORCID 0000-0003-2111-507X), Pollard, Benjamin (ORCID 0000-0002-5109-6415), Aiken, John M. (ORCID 0000-0003-0717-4583), Lewandowski, H. J.
Source: Physical Review Physics Education Research. Jan-Jun 2022 18(1).
Availability: American Physical Society. One Physics Ellipse 4th Floor, College Park, MD 20740-3844. Tel: 301-209-3200; Fax: 301-209-0865; e-mail: assocpub@aps.org; Web site: http://prst-per.aps.org
Peer Reviewed: Y
Page Count: 16
Publication Date: 2022
Sponsoring Agency: National Science Foundation (NSF)
Contract Number: PHY1734006
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Natural Language Processing, Science Education, Physics, Artificial Intelligence, Models, Data Analysis, Classification, Student Reaction, Test Format, College Students
Geographic Terms: Colorado (Boulder)
DOI: 10.1103/PhysRevPhysEducRes.18.010141
ISSN: 2469-9896
Abstract: Surveys have long been used in physics education research to understand student reasoning and inform course improvements. However, to make analysis of large sets of responses practical, most surveys use a closed-response format with a small set of potential responses. Open-ended formats, such as written free response, can provide deeper insights into student thinking, but take much longer to analyze, especially with a large number of responses. Here, we explore natural language processing as a computational solution to this problem. We create a machine learning model that can take student responses from the Physics Measurement Questionnaire as input, and output a categorization of student reasoning based on different reasoning paradigms. Our model yields classifications with the same level of agreement as that between two humans categorizing the data, but can be done by a computer, and thus can be scaled for large datasets. In this work, we describe the algorithms and methodologies used to create, train, and test our natural language processing system. We also present the results of the analysis and discuss the utility of these approaches for analyzing open-response data in education research.
Abstractor: As Provided
Entry Date: 2022
Accession Number: EJ1355094
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1355094
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Classification of Open-Ended Responses to a Research-Based Assessment Using Natural Language Processing
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Wilson%2C+Joseph%22">Wilson, Joseph</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-2111-507X">0000-0003-2111-507X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Pollard%2C+Benjamin%22">Pollard, Benjamin</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-5109-6415">0000-0002-5109-6415</externalLink>)<br /><searchLink fieldCode="AR" term="%22Aiken%2C+John+M%2E%22">Aiken, John M.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-0717-4583">0000-0003-0717-4583</externalLink>)<br /><searchLink fieldCode="AR" term="%22Lewandowski%2C+H%2E+J%2E%22">Lewandowski, H. J.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Physical+Review+Physics+Education+Research%22"><i>Physical Review Physics Education Research</i></searchLink>. Jan-Jun 2022 18(1).
– Name: Avail
  Label: Availability
  Group: Avail
  Data: American Physical Society. One Physics Ellipse 4th Floor, College Park, MD 20740-3844. Tel: 301-209-3200; Fax: 301-209-0865; e-mail: assocpub@aps.org; Web site: http://prst-per.aps.org
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 16
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Science Foundation (NSF)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: PHY1734006
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Science+Education%22">Science Education</searchLink><br /><searchLink fieldCode="DE" term="%22Physics%22">Physics</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Models%22">Models</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Analysis%22">Data Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Classification%22">Classification</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Reaction%22">Student Reaction</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Format%22">Test Format</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Colorado+%28Boulder%29%22">Colorado (Boulder)</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1103/PhysRevPhysEducRes.18.010141
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2469-9896
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Surveys have long been used in physics education research to understand student reasoning and inform course improvements. However, to make analysis of large sets of responses practical, most surveys use a closed-response format with a small set of potential responses. Open-ended formats, such as written free response, can provide deeper insights into student thinking, but take much longer to analyze, especially with a large number of responses. Here, we explore natural language processing as a computational solution to this problem. We create a machine learning model that can take student responses from the Physics Measurement Questionnaire as input, and output a categorization of student reasoning based on different reasoning paradigms. Our model yields classifications with the same level of agreement as that between two humans categorizing the data, but can be done by a computer, and thus can be scaled for large datasets. In this work, we describe the algorithms and methodologies used to create, train, and test our natural language processing system. We also present the results of the analysis and discuss the utility of these approaches for analyzing open-response data in education research.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2022
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1355094
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1355094
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1103/PhysRevPhysEducRes.18.010141
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 16
    Subjects:
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Science Education
        Type: general
      – SubjectFull: Physics
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Models
        Type: general
      – SubjectFull: Data Analysis
        Type: general
      – SubjectFull: Classification
        Type: general
      – SubjectFull: Student Reaction
        Type: general
      – SubjectFull: Test Format
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Colorado (Boulder)
        Type: general
    Titles:
      – TitleFull: Classification of Open-Ended Responses to a Research-Based Assessment Using Natural Language Processing
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Wilson, Joseph
      – PersonEntity:
          Name:
            NameFull: Pollard, Benjamin
      – PersonEntity:
          Name:
            NameFull: Aiken, John M.
      – PersonEntity:
          Name:
            NameFull: Lewandowski, H. J.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-electronic
              Value: 2469-9896
          Numbering:
            – Type: volume
              Value: 18
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Physical Review Physics Education Research
              Type: main
ResultId 1