View in EDS

Classification of Reflective Writing: A Comparative Analysis with Shallow Machine Learning and Pre-Trained Language Models

Saved in:
Bibliographic Details
Title: Classification of Reflective Writing: A Comparative Analysis with Shallow Machine Learning and Pre-Trained Language Models
Language: English
Authors: Chengming Zhang (ORCID 0009-0007-8695-5455), Florian Hofmann, Lea Plößl, Michaela Gläser-Zikuda
Source: Education and Information Technologies. 2024 29(16):21593-21619.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 27
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Foreign Countries, Higher Education, Reflection, Student Writing Models, Preservice Teacher Education, Writing (Composition), Artificial Intelligence, Comparative Analysis, Writing Evaluation, Automation
Geographic Terms: Germany
DOI: 10.1007/s10639-024-12720-0
ISSN: 1360-2357
1573-7608
Abstract: Reflective practice holds critical importance, for example, in higher education and teacher education, yet promoting students' reflective skills has been a persistent challenge. The emergence of revolutionary artificial intelligence technologies, notably in machine learning and large language models, heralds potential breakthroughs in this domain. The current research on analyzing reflective writing hinges on sentence-level classification. Such an approach, however, may fall short of providing a holistic grasp of written reflection. Therefore, this study employs shallow machine learning algorithms and pre-trained language models, namely BERT, RoBERTa, BigBird, and Longformer, with the intention of enhancing the document-level classification accuracy of reflective writings. A dataset of 1,043 reflective writings was collected in a teacher education program at a German university (M = 251.38 words, SD = 143.08 words). Our findings indicated that BigBird and Longformer models significantly outperformed BERT and RoBERTa, achieving classification accuracies of 76.26% and 77.22%, respectively, with less than 60% accuracy observed in shallow machine learning models. The outcomes of this study contribute to refining document-level classification of reflective writings and have implications for augmenting automated feedback mechanisms in teacher education.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1450565
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1450565
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Classification of Reflective Writing: A Comparative Analysis with Shallow Machine Learning and Pre-Trained Language Models
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Chengming+Zhang%22">Chengming Zhang</searchLink> (ORCID <externalLink term="http://orcid.org/0009-0007-8695-5455">0009-0007-8695-5455</externalLink>)<br /><searchLink fieldCode="AR" term="%22Florian+Hofmann%22">Florian Hofmann</searchLink><br /><searchLink fieldCode="AR" term="%22Lea+Plößl%22">Lea Plößl</searchLink><br /><searchLink fieldCode="AR" term="%22Michaela+Gläser-Zikuda%22">Michaela Gläser-Zikuda</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Education+and+Information+Technologies%22"><i>Education and Information Technologies</i></searchLink>. 2024 29(16):21593-21619.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 27
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="DE" term="%22Reflection%22">Reflection</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Writing+Models%22">Student Writing Models</searchLink><br /><searchLink fieldCode="DE" term="%22Preservice+Teacher+Education%22">Preservice Teacher Education</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+%28Composition%29%22">Writing (Composition)</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Evaluation%22">Writing Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Germany%22">Germany</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s10639-024-12720-0
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1360-2357<br />1573-7608
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Reflective practice holds critical importance, for example, in higher education and teacher education, yet promoting students' reflective skills has been a persistent challenge. The emergence of revolutionary artificial intelligence technologies, notably in machine learning and large language models, heralds potential breakthroughs in this domain. The current research on analyzing reflective writing hinges on sentence-level classification. Such an approach, however, may fall short of providing a holistic grasp of written reflection. Therefore, this study employs shallow machine learning algorithms and pre-trained language models, namely BERT, RoBERTa, BigBird, and Longformer, with the intention of enhancing the document-level classification accuracy of reflective writings. A dataset of 1,043 reflective writings was collected in a teacher education program at a German university (M = 251.38 words, SD = 143.08 words). Our findings indicated that BigBird and Longformer models significantly outperformed BERT and RoBERTa, achieving classification accuracies of 76.26% and 77.22%, respectively, with less than 60% accuracy observed in shallow machine learning models. The outcomes of this study contribute to refining document-level classification of reflective writings and have implications for augmenting automated feedback mechanisms in teacher education.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1450565
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1450565
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10639-024-12720-0
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 27
        StartPage: 21593
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Higher Education
        Type: general
      – SubjectFull: Reflection
        Type: general
      – SubjectFull: Student Writing Models
        Type: general
      – SubjectFull: Preservice Teacher Education
        Type: general
      – SubjectFull: Writing (Composition)
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Writing Evaluation
        Type: general
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Germany
        Type: general
    Titles:
      – TitleFull: Classification of Reflective Writing: A Comparative Analysis with Shallow Machine Learning and Pre-Trained Language Models
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Chengming Zhang
      – PersonEntity:
          Name:
            NameFull: Florian Hofmann
      – PersonEntity:
          Name:
            NameFull: Lea Plößl
      – PersonEntity:
          Name:
            NameFull: Michaela Gläser-Zikuda
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 11
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 1360-2357
            – Type: issn-electronic
              Value: 1573-7608
          Numbering:
            – Type: volume
              Value: 29
            – Type: issue
              Value: 16
          Titles:
            – TitleFull: Education and Information Technologies
              Type: main
ResultId 1