Using Machine Learning to Detect SMART Model Cognitive Operations in Mathematical Problem-Solving Process
Saved in:
| Title: | Using Machine Learning to Detect SMART Model Cognitive Operations in Mathematical Problem-Solving Process |
|---|---|
| Language: | English |
| Authors: | Zhang, Jiayi, Andres, Juliana Ma. Alexandra L., Hutt, Stephen (ORCID |
| Source: | Journal of Educational Data Mining. 2022 14(3):76-108. |
| Availability: | International Educational Data Mining. e-mail: jedm.editor@gmail.com; Web site: https://jedm.educationaldatamining.org/index.php/JEDM |
| Peer Reviewed: | Y |
| Page Count: | 33 |
| Publication Date: | 2022 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Elementary Education Grade 6 Intermediate Grades Middle Schools Grade 7 Junior High Schools Secondary Education |
| Descriptors: | Problem Solving, Mathematics Instruction, Learning Management Systems, Learning Analytics, Scaffolding (Teaching Technique), Metacognition, Attention Control, Protocol Analysis, Computer Mediated Communication, Models, Generalization, Reliability, Cognitive Processes, Learning Strategies, Questionnaires, Algorithms, Grade 6, Grade 7 |
| Assessment and Survey Identifiers: | Motivated Strategies for Learning Questionnaire |
| ISSN: | 2157-2100 |
| Abstract: | Self-regulated learning (SRL) is a critical component of mathematics problem-solving. Students skilled in SRL are more likely to effectively set goals, search for information, and direct their attention and cognitive process so that they align their efforts with their objectives. An influential framework for SRL, the SMART model (Winne, 2017), proposes that five cognitive operations (i.e., searching, monitoring, assembling, rehearsing, and translating) play a key role in SRL. However, these categories encompass a wide range of behaviors, making measurement challenging -- often involving observing individual students and recording their think-aloud activities or asking students to complete labor-intensive tagging activities as they work. In the current study, to achieve better scalability, we operationalized indicators of SMART operations and developed automated detectors using machine learning. We analyzed students' textual responses and interaction data collected from a mathematical learning platform where students are asked to thoroughly explain their solutions and are scaffolded in communicating their problem-solving process. Due to the rarity in data for one of the seven SRL indicators operationalized, we built six models to reflect students' use of four SMART operations. These models are found to be reliable and generalizable, with AUC ROCs ranging from 0.76-0.89. When applied to the full test set, these detectors are relatively robust to algorithmic bias, performing well across different student populations and with no consistent bias against a specific group of students. |
| Abstractor: | As Provided |
| Entry Date: | 2023 |
| Accession Number: | EJ1373133 |
| Database: | ERIC |
| Abstract: | Self-regulated learning (SRL) is a critical component of mathematics problem-solving. Students skilled in SRL are more likely to effectively set goals, search for information, and direct their attention and cognitive process so that they align their efforts with their objectives. An influential framework for SRL, the SMART model (Winne, 2017), proposes that five cognitive operations (i.e., searching, monitoring, assembling, rehearsing, and translating) play a key role in SRL. However, these categories encompass a wide range of behaviors, making measurement challenging -- often involving observing individual students and recording their think-aloud activities or asking students to complete labor-intensive tagging activities as they work. In the current study, to achieve better scalability, we operationalized indicators of SMART operations and developed automated detectors using machine learning. We analyzed students' textual responses and interaction data collected from a mathematical learning platform where students are asked to thoroughly explain their solutions and are scaffolded in communicating their problem-solving process. Due to the rarity in data for one of the seven SRL indicators operationalized, we built six models to reflect students' use of four SMART operations. These models are found to be reliable and generalizable, with AUC ROCs ranging from 0.76-0.89. When applied to the full test set, these detectors are relatively robust to algorithmic bias, performing well across different student populations and with no consistent bias against a specific group of students. |
|---|---|
| ISSN: | 2157-2100 |
