Classification of Reflective Writing: A Comparative Analysis with Shallow Machine Learning and Pre-Trained Language Models
Saved in:
| Title: | Classification of Reflective Writing: A Comparative Analysis with Shallow Machine Learning and Pre-Trained Language Models |
|---|---|
| Language: | English |
| Authors: | Chengming Zhang (ORCID |
| Source: | Education and Information Technologies. 2024 29(16):21593-21619. |
| Availability: | Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/ |
| Peer Reviewed: | Y |
| Page Count: | 27 |
| Publication Date: | 2024 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Foreign Countries, Higher Education, Reflection, Student Writing Models, Preservice Teacher Education, Writing (Composition), Artificial Intelligence, Comparative Analysis, Writing Evaluation, Automation |
| Geographic Terms: | Germany |
| DOI: | 10.1007/s10639-024-12720-0 |
| ISSN: | 1360-2357 1573-7608 |
| Abstract: | Reflective practice holds critical importance, for example, in higher education and teacher education, yet promoting students' reflective skills has been a persistent challenge. The emergence of revolutionary artificial intelligence technologies, notably in machine learning and large language models, heralds potential breakthroughs in this domain. The current research on analyzing reflective writing hinges on sentence-level classification. Such an approach, however, may fall short of providing a holistic grasp of written reflection. Therefore, this study employs shallow machine learning algorithms and pre-trained language models, namely BERT, RoBERTa, BigBird, and Longformer, with the intention of enhancing the document-level classification accuracy of reflective writings. A dataset of 1,043 reflective writings was collected in a teacher education program at a German university (M = 251.38 words, SD = 143.08 words). Our findings indicated that BigBird and Longformer models significantly outperformed BERT and RoBERTa, achieving classification accuracies of 76.26% and 77.22%, respectively, with less than 60% accuracy observed in shallow machine learning models. The outcomes of this study contribute to refining document-level classification of reflective writings and have implications for augmenting automated feedback mechanisms in teacher education. |
| Abstractor: | As Provided |
| Entry Date: | 2024 |
| Accession Number: | EJ1450565 |
| Database: | ERIC |
| Abstract: | Reflective practice holds critical importance, for example, in higher education and teacher education, yet promoting students' reflective skills has been a persistent challenge. The emergence of revolutionary artificial intelligence technologies, notably in machine learning and large language models, heralds potential breakthroughs in this domain. The current research on analyzing reflective writing hinges on sentence-level classification. Such an approach, however, may fall short of providing a holistic grasp of written reflection. Therefore, this study employs shallow machine learning algorithms and pre-trained language models, namely BERT, RoBERTa, BigBird, and Longformer, with the intention of enhancing the document-level classification accuracy of reflective writings. A dataset of 1,043 reflective writings was collected in a teacher education program at a German university (M = 251.38 words, SD = 143.08 words). Our findings indicated that BigBird and Longformer models significantly outperformed BERT and RoBERTa, achieving classification accuracies of 76.26% and 77.22%, respectively, with less than 60% accuracy observed in shallow machine learning models. The outcomes of this study contribute to refining document-level classification of reflective writings and have implications for augmenting automated feedback mechanisms in teacher education. |
|---|---|
| ISSN: | 1360-2357 1573-7608 |
| DOI: | 10.1007/s10639-024-12720-0 |
