Constructing and Validating a Q-Matrix for Cognitive Diagnostic Analysis of the Listening Comprehension Section of the IELTS
Saved in:
| Title: | Constructing and Validating a Q-Matrix for Cognitive Diagnostic Analysis of the Listening Comprehension Section of the IELTS |
|---|---|
| Language: | English |
| Authors: | Seyedeh Azadeh Ghiasian, Fatemeh Hemmati, Seyyed Mohammad Alavi, Afsar Rouhi |
| Source: | International Journal of Language Testing. 2025 15(1):54-76. |
| Availability: | Tabaran Institute of Higher Education. Shariati 60, Shariati Blvd, Ghasem Abad, Mashhad, Khorasan Razavi, Iran. Tel: +98 (51) 35227215; e-mail: ijlt@tabaran.ac.ir; Web site: http://www.ijlt.ir/ |
| Peer Reviewed: | Y |
| Page Count: | 23 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education Adult Education |
| Descriptors: | Test Items, Listening Comprehension Tests, English (Second Language), Language Tests, Second Language Learning, Foreign Countries, Intonation, Suprasegmentals, Test Validity, Item Analysis, Phrase Structure, Language Processing, Inferences, Goodness of Fit, College Students, Adult Students, Auditory Discrimination |
| Geographic Terms: | Iran |
| Assessment and Survey Identifiers: | International English Language Testing System |
| ISSN: | 2476-5880 |
| Abstract: | A critical component of cognitive diagnostic models (CDMs) is a Q-matrix that stipulates associations between items of a test and their required attributes. The present study aims to develop and empirically validate a Q-matrix for the listening comprehension section of the International English Language Testing System (IELTS). To this end, a listening comprehension test of the IELTS was administered to 820 Iranian test takers. According to theories, taxonomies, and models of second/foreign language (L2) listening comprehension, previous studies on the utility of CDMs to L2 listening comprehension, detailed content analysis of the test items, and consultation with several content experts, an initial Q-matrix was first developed. Through the technique suggested by de la Torre and Chiu (2016), along with checking heatmap plots and mesa plots using the GDINA package in R, the Q-matrix was then empirically validated. Generally, six attributes were extracted for the listening section, namely, (1) Linguistic knowledge (LKA), (2) understanding prosodic patterns (UPP), (3) ability to understand and make paraphrases (PAR), (4) ability to understand specific factual information such as names, numbers, and so forth (UFI), (5) ability to understand explicit information (UEI), and (6) ability to make inference (INF). Finally, the results of the fit of the GDINA model to the data, at both item and test levels, indicated the adequate model-data fit and the plausibility of the Q-matrix. The implications of the study were also discussed. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1463868 |
| Database: | ERIC |
| Abstract: | A critical component of cognitive diagnostic models (CDMs) is a Q-matrix that stipulates associations between items of a test and their required attributes. The present study aims to develop and empirically validate a Q-matrix for the listening comprehension section of the International English Language Testing System (IELTS). To this end, a listening comprehension test of the IELTS was administered to 820 Iranian test takers. According to theories, taxonomies, and models of second/foreign language (L2) listening comprehension, previous studies on the utility of CDMs to L2 listening comprehension, detailed content analysis of the test items, and consultation with several content experts, an initial Q-matrix was first developed. Through the technique suggested by de la Torre and Chiu (2016), along with checking heatmap plots and mesa plots using the GDINA package in R, the Q-matrix was then empirically validated. Generally, six attributes were extracted for the listening section, namely, (1) Linguistic knowledge (LKA), (2) understanding prosodic patterns (UPP), (3) ability to understand and make paraphrases (PAR), (4) ability to understand specific factual information such as names, numbers, and so forth (UFI), (5) ability to understand explicit information (UEI), and (6) ability to make inference (INF). Finally, the results of the fit of the GDINA model to the data, at both item and test levels, indicated the adequate model-data fit and the plausibility of the Q-matrix. The implications of the study were also discussed. |
|---|---|
| ISSN: | 2476-5880 |
