ReQUESTA: A Hybrid Agentic Framework for Generating Cognitively Diverse Multiple-Choice Questions
Saved in:
| Title: | ReQUESTA: A Hybrid Agentic Framework for Generating Cognitively Diverse Multiple-Choice Questions |
|---|---|
| Language: | English |
| Authors: | Yu Tian, Shubham Chakraborty, Linh Huynh (ORCID |
| Source: | Grantee Submission. 2026. |
| Peer Reviewed: | Y |
| Page Count: | 11 |
| Publication Date: | 2026 |
| Sponsoring Agency: | National Center for Education Research (NCER) (ED/IES) |
| Contract Number: | R305T240035 |
| Document Type: | Speeches/Meeting Papers Reports - Research |
| Descriptors: | Test Construction, Multiple Choice Tests, Artificial Intelligence, Natural Language Processing, Computer Assisted Testing, Classification, Thinking Skills |
| DOI: | 10.59668/2551.25167 |
| Abstract: | This study presents ReQUESTA, a hybrid agentic framework that integrates LLM-powered and rule-based agents to generate multiple choice questions (MCQs) with distinct cognitive focuses: text-based, inferential, and main-idea. To provide an initial validation of the framework, expert raters evaluated 100 ReQUESTA-generated MCQs using a cognitive classification rubric to assess alignment between intended and perceived cognitive categories. The results indicate a high level of agreement between system-assigned and expert-assigned labels (agreement rate = 0.95), suggesting that ReQUESTA can reliably instantiate targeted cognitive distinctions in question generation. These findings offer preliminary evidence of the framework's capacity to support cognitively diverse and pedagogically meaningful assessment design. ReQUESTA's modular and hybrid design supports scalable, iterative, and evidence-based assessment development. Future work will include psychometric validation and expansion to additional cognitive categories such as application-level questions. [This paper was published in: "Proceedings of the Learning Engineering Research Network Convening (LERN 2026)," 2026, pp. 303-312.] |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2026 |
| Accession Number: | ED678904 |
| Database: | ERIC |
| Abstract: | This study presents ReQUESTA, a hybrid agentic framework that integrates LLM-powered and rule-based agents to generate multiple choice questions (MCQs) with distinct cognitive focuses: text-based, inferential, and main-idea. To provide an initial validation of the framework, expert raters evaluated 100 ReQUESTA-generated MCQs using a cognitive classification rubric to assess alignment between intended and perceived cognitive categories. The results indicate a high level of agreement between system-assigned and expert-assigned labels (agreement rate = 0.95), suggesting that ReQUESTA can reliably instantiate targeted cognitive distinctions in question generation. These findings offer preliminary evidence of the framework's capacity to support cognitively diverse and pedagogically meaningful assessment design. ReQUESTA's modular and hybrid design supports scalable, iterative, and evidence-based assessment development. Future work will include psychometric validation and expansion to additional cognitive categories such as application-level questions. [This paper was published in: "Proceedings of the Learning Engineering Research Network Convening (LERN 2026)," 2026, pp. 303-312.] |
|---|---|
| DOI: | 10.59668/2551.25167 |
