Diagnostic Classification Model for Forced-Choice Items and Noncognitive Tests
Saved in:
| Title: | Diagnostic Classification Model for Forced-Choice Items and Noncognitive Tests |
|---|---|
| Language: | English |
| Authors: | Huang, Hung-Yu (ORCID |
| Source: | Educational and Psychological Measurement. Feb 2023 83(1):146-180. |
| Availability: | SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com |
| Peer Reviewed: | Y |
| Page Count: | 35 |
| Publication Date: | 2023 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Test Items, Classification, Bayesian Statistics, Decision Making, Models, Guidelines, Response Style (Tests), Simulation, Comparative Analysis, Accuracy, Item Response Theory, Test Format, Psychological Patterns, Diagnostic Tests |
| DOI: | 10.1177/00131644211069906 |
| ISSN: | 0013-1644 1552-3888 |
| Abstract: | The forced-choice (FC) item formats used for noncognitive tests typically develop a set of response options that measure different traits and instruct respondents to make judgments among these options in terms of their preference to control the response biases that are commonly observed in normative tests. Diagnostic classification models (DCMs) can provide information regarding the mastery status of test takers on latent discrete variables and are more commonly used for cognitive tests employed in educational settings than for noncognitive tests. The purpose of this study is to develop a new class of DCM for FC items under the higher-order DCM framework to meet the practical demands of simultaneously controlling for response biases and providing diagnostic classification information. By conducting a series of simulations and calibrating the model parameters with a Bayesian estimation, the study shows that, in general, the model parameters can be recovered satisfactorily with the use of long tests and large samples. More attributes improve the precision of the second-order latent trait estimation in a long test, but decrease the classification accuracy and the estimation quality of the structural parameters. When statements are allowed to load on two distinct attributes in paired comparison items, the specific-attribute condition produces better a parameter estimation than the overlap-attribute condition. Finally, an empirical analysis related to work-motivation measures is presented to demonstrate the applications and implications of the new model. |
| Abstractor: | As Provided |
| Entry Date: | 2023 |
| Accession Number: | EJ1360527 |
| Database: | ERIC |
| Abstract: | The forced-choice (FC) item formats used for noncognitive tests typically develop a set of response options that measure different traits and instruct respondents to make judgments among these options in terms of their preference to control the response biases that are commonly observed in normative tests. Diagnostic classification models (DCMs) can provide information regarding the mastery status of test takers on latent discrete variables and are more commonly used for cognitive tests employed in educational settings than for noncognitive tests. The purpose of this study is to develop a new class of DCM for FC items under the higher-order DCM framework to meet the practical demands of simultaneously controlling for response biases and providing diagnostic classification information. By conducting a series of simulations and calibrating the model parameters with a Bayesian estimation, the study shows that, in general, the model parameters can be recovered satisfactorily with the use of long tests and large samples. More attributes improve the precision of the second-order latent trait estimation in a long test, but decrease the classification accuracy and the estimation quality of the structural parameters. When statements are allowed to load on two distinct attributes in paired comparison items, the specific-attribute condition produces better a parameter estimation than the overlap-attribute condition. Finally, an empirical analysis related to work-motivation measures is presented to demonstrate the applications and implications of the new model. |
|---|---|
| ISSN: | 0013-1644 1552-3888 |
| DOI: | 10.1177/00131644211069906 |
