Design and Validation of the Metacognitive Diagnostic Disruption in AI Decision-Making Scale (MDD-AI Scale) among Educational Administrators in Jordan: Insights from the Network Analysis Perspective
Saved in:
| Title: | Design and Validation of the Metacognitive Diagnostic Disruption in AI Decision-Making Scale (MDD-AI Scale) among Educational Administrators in Jordan: Insights from the Network Analysis Perspective |
|---|---|
| Language: | English |
| Authors: | Yusra jadallah abed Khasawneh (ORCID |
| Source: | Journal of Computer Assisted Learning. 2026 42(2). |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 25 |
| Publication Date: | 2026 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Metacognition, Artificial Intelligence, Measures (Individuals), Educational Administration, Test Construction, Test Validity, Decision Making, Foreign Countries, Self Management, Test Reliability, Technology Uses in Education, Administrator Attitudes |
| Geographic Terms: | Jordan |
| DOI: | 10.1002/jcal.70216 |
| ISSN: | 0266-4909 1365-2729 |
| Abstract: | Background: In recent decades, emerging technologies--particularly Artificial Intelligence (AI)--have had a profound impact on the structure and decision-making processes within educational systems. AI-based tools now play a central role in data analysis, performance prediction, and personalised learning design. Objectives: This study aimed to design and validate the Metacognitive Diagnostic Disruption in AI Decision-Making Scale (MDD-AI Scale) for educational administrators, focusing on metacognitive disruptions--disturbances in self-monitoring and self-regulation processes caused by AI involvement in decision-making. Methods: Using an exploratory mixed-methods design, the study combined a literature review and interviews with 20 Jordanian educational administrators to develop 52 items from an initial 92. Data from 670 administrators were analysed for validity and reliability, with exploratory graph analysis (EGA) providing a network-based perspective. Results and Conclusions: Exploratory Factor Analysis (EFA) revealed a six-factor structure--Metacognitive Self-Monitoring Disruption, Metacognitive Self-Evaluation Disruption, Cognitive Flexibility Reduction, Cognitive Overreliance on AI, Conflict Detection Shutdown, and Disruption of Metacognitive Decision Planning--explaining 67.6% of the variance. Confirmatory Factor Analysis (CFA) showed good fit (RMSEA = 0.072, CFI = 0.918), with AVE values > 0.50 confirming convergent validity. Reliability was high across all indices (α = 0.904--0.951; ω = 0.897--0.952; CR > 0.89; ICC = 0.752--0.870). Measurement invariance across gender indicated structural equivalence. Exploratory Graph Analysis (EGA) and Random Forest Modeling (RFM) further supported robustness and predictive utility. Overall, the 42-item MDD-AI Scale is a valid, reliable, and innovative tool for assessing metacognitive disruptions in algorithmic decision-making within educational management. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1500592 |
| Database: | ERIC |
| Abstract: | Background: In recent decades, emerging technologies--particularly Artificial Intelligence (AI)--have had a profound impact on the structure and decision-making processes within educational systems. AI-based tools now play a central role in data analysis, performance prediction, and personalised learning design. Objectives: This study aimed to design and validate the Metacognitive Diagnostic Disruption in AI Decision-Making Scale (MDD-AI Scale) for educational administrators, focusing on metacognitive disruptions--disturbances in self-monitoring and self-regulation processes caused by AI involvement in decision-making. Methods: Using an exploratory mixed-methods design, the study combined a literature review and interviews with 20 Jordanian educational administrators to develop 52 items from an initial 92. Data from 670 administrators were analysed for validity and reliability, with exploratory graph analysis (EGA) providing a network-based perspective. Results and Conclusions: Exploratory Factor Analysis (EFA) revealed a six-factor structure--Metacognitive Self-Monitoring Disruption, Metacognitive Self-Evaluation Disruption, Cognitive Flexibility Reduction, Cognitive Overreliance on AI, Conflict Detection Shutdown, and Disruption of Metacognitive Decision Planning--explaining 67.6% of the variance. Confirmatory Factor Analysis (CFA) showed good fit (RMSEA = 0.072, CFI = 0.918), with AVE values > 0.50 confirming convergent validity. Reliability was high across all indices (α = 0.904--0.951; ω = 0.897--0.952; CR > 0.89; ICC = 0.752--0.870). Measurement invariance across gender indicated structural equivalence. Exploratory Graph Analysis (EGA) and Random Forest Modeling (RFM) further supported robustness and predictive utility. Overall, the 42-item MDD-AI Scale is a valid, reliable, and innovative tool for assessing metacognitive disruptions in algorithmic decision-making within educational management. |
|---|---|
| ISSN: | 0266-4909 1365-2729 |
| DOI: | 10.1002/jcal.70216 |
