Exploring the Influence of Response Time Allocation on Item Revisiting: Implications for Test-Taking Strategies in Cognitive Diagnostic Assessments
Saved in:
| Title: | Exploring the Influence of Response Time Allocation on Item Revisiting: Implications for Test-Taking Strategies in Cognitive Diagnostic Assessments |
|---|---|
| Language: | English |
| Authors: | Ziyuan Zhao (ORCID |
| Source: | Journal of Educational Measurement. 2026 63(1). |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 27 |
| Publication Date: | 2026 |
| Document Type: | Journal Articles Reports - Evaluative |
| Descriptors: | Item Response Theory, Test Items, Test Wiseness, Cognitive Measurement, Diagnostic Tests, Computer Assisted Testing, Reaction Time, Accuracy, Markov Processes, Monte Carlo Methods, Bayesian Statistics |
| DOI: | 10.1111/jedm.70021 |
| ISSN: | 0022-0655 1745-3984 |
| Abstract: | Computer-based assessments offer readily available process data for analysis to gain a deeper understanding of the response process. A common response strategy is item revisiting, which can reduce examinees' anxiety and improve their chances of answering questions correctly, and data on item revisiting are recorded automatically in system logs. The approach reported here is to combine two useful and easily accessible types of process data--item response times and item-revisiting data--with a cognitive diagnostic model to enhance accuracy, identify examinees' level of mastery in specific skills within a particular knowledge domain, and provide personalized diagnostic feedback. The modeling involves two monotonicity hypotheses: (1) examinees who engaged in more revisiting in previous items are more likely to revisit the current item; (2) a longer accumulated response time on previous items results in less remaining time, reducing the likelihood of revisiting the current item. Unlike previous studies in which response time was modeled separately, the focus here is on examinees' revisiting behavior, thus the response time is included in the revisiting modeling as a covariate. This allows an in-depth investigation of how the accumulated response time influences revisiting behavior, as well as an exploration of the relationship between response strategy (i.e., item revisiting) and time allocation. The Markov-chain Monte Carlo approach is used for parameter estimation, and its effectiveness is evaluated using two Bayesian evaluation criteria based on posterior samples. Simulation results show that this method is effective for recovering parameters, and an example analysis verifies the the proposed model. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1501280 |
| Database: | ERIC |
| Abstract: | Computer-based assessments offer readily available process data for analysis to gain a deeper understanding of the response process. A common response strategy is item revisiting, which can reduce examinees' anxiety and improve their chances of answering questions correctly, and data on item revisiting are recorded automatically in system logs. The approach reported here is to combine two useful and easily accessible types of process data--item response times and item-revisiting data--with a cognitive diagnostic model to enhance accuracy, identify examinees' level of mastery in specific skills within a particular knowledge domain, and provide personalized diagnostic feedback. The modeling involves two monotonicity hypotheses: (1) examinees who engaged in more revisiting in previous items are more likely to revisit the current item; (2) a longer accumulated response time on previous items results in less remaining time, reducing the likelihood of revisiting the current item. Unlike previous studies in which response time was modeled separately, the focus here is on examinees' revisiting behavior, thus the response time is included in the revisiting modeling as a covariate. This allows an in-depth investigation of how the accumulated response time influences revisiting behavior, as well as an exploration of the relationship between response strategy (i.e., item revisiting) and time allocation. The Markov-chain Monte Carlo approach is used for parameter estimation, and its effectiveness is evaluated using two Bayesian evaluation criteria based on posterior samples. Simulation results show that this method is effective for recovering parameters, and an example analysis verifies the the proposed model. |
|---|---|
| ISSN: | 0022-0655 1745-3984 |
| DOI: | 10.1111/jedm.70021 |
