View in EDS

ReQUESTA: A Hybrid Agentic Framework for Generating Cognitively Diverse Multiple-Choice Questions

Saved in:
Bibliographic Details
Title: ReQUESTA: A Hybrid Agentic Framework for Generating Cognitively Diverse Multiple-Choice Questions
Language: English
Authors: Yu Tian, Shubham Chakraborty, Linh Huynh (ORCID 0000-0002-5387-4137), Katerina Christhilf (ORCID 0000-0003-3901-8665), Micah Watanabe (ORCID 0000-0002-9929-6600), Danielle S. McNamara (ORCID 0000-0001-5869-1420)
Source: Grantee Submission. 2026.
Peer Reviewed: Y
Page Count: 11
Publication Date: 2026
Sponsoring Agency: National Center for Education Research (NCER) (ED/IES)
Contract Number: R305T240035
Document Type: Speeches/Meeting Papers
Reports - Research
Descriptors: Test Construction, Multiple Choice Tests, Artificial Intelligence, Natural Language Processing, Computer Assisted Testing, Classification, Thinking Skills
DOI: 10.59668/2551.25167
Abstract: This study presents ReQUESTA, a hybrid agentic framework that integrates LLM-powered and rule-based agents to generate multiple choice questions (MCQs) with distinct cognitive focuses: text-based, inferential, and main-idea. To provide an initial validation of the framework, expert raters evaluated 100 ReQUESTA-generated MCQs using a cognitive classification rubric to assess alignment between intended and perceived cognitive categories. The results indicate a high level of agreement between system-assigned and expert-assigned labels (agreement rate = 0.95), suggesting that ReQUESTA can reliably instantiate targeted cognitive distinctions in question generation. These findings offer preliminary evidence of the framework's capacity to support cognitively diverse and pedagogically meaningful assessment design. ReQUESTA's modular and hybrid design supports scalable, iterative, and evidence-based assessment development. Future work will include psychometric validation and expansion to additional cognitive categories such as application-level questions. [This paper was published in: "Proceedings of the Learning Engineering Research Network Convening (LERN 2026)," 2026, pp. 303-312.]
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2026
Accession Number: ED678904
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: ED678904
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: ReQUESTA: A Hybrid Agentic Framework for Generating Cognitively Diverse Multiple-Choice Questions
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Yu+Tian%22">Yu Tian</searchLink><br /><searchLink fieldCode="AR" term="%22Shubham+Chakraborty%22">Shubham Chakraborty</searchLink><br /><searchLink fieldCode="AR" term="%22Linh+Huynh%22">Linh Huynh</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-5387-4137">0000-0002-5387-4137</externalLink>)<br /><searchLink fieldCode="AR" term="%22Katerina+Christhilf%22">Katerina Christhilf</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3901-8665">0000-0003-3901-8665</externalLink>)<br /><searchLink fieldCode="AR" term="%22Micah+Watanabe%22">Micah Watanabe</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-9929-6600">0000-0002-9929-6600</externalLink>)<br /><searchLink fieldCode="AR" term="%22Danielle+S%2E+McNamara%22">Danielle S. McNamara</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5869-1420">0000-0001-5869-1420</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Grantee+Submission%22"><i>Grantee Submission</i></searchLink>. 2026.
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 11
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2026
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Center for Education Research (NCER) (ED/IES)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305T240035
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Test+Construction%22">Test Construction</searchLink><br /><searchLink fieldCode="DE" term="%22Multiple+Choice+Tests%22">Multiple Choice Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Classification%22">Classification</searchLink><br /><searchLink fieldCode="DE" term="%22Thinking+Skills%22">Thinking Skills</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.59668/2551.25167
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This study presents ReQUESTA, a hybrid agentic framework that integrates LLM-powered and rule-based agents to generate multiple choice questions (MCQs) with distinct cognitive focuses: text-based, inferential, and main-idea. To provide an initial validation of the framework, expert raters evaluated 100 ReQUESTA-generated MCQs using a cognitive classification rubric to assess alignment between intended and perceived cognitive categories. The results indicate a high level of agreement between system-assigned and expert-assigned labels (agreement rate = 0.95), suggesting that ReQUESTA can reliably instantiate targeted cognitive distinctions in question generation. These findings offer preliminary evidence of the framework's capacity to support cognitively diverse and pedagogically meaningful assessment design. ReQUESTA's modular and hybrid design supports scalable, iterative, and evidence-based assessment development. Future work will include psychometric validation and expansion to additional cognitive categories such as application-level questions. [This paper was published in: "Proceedings of the Learning Engineering Research Network Convening (LERN 2026)," 2026, pp. 303-312.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED678904
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED678904
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.59668/2551.25167
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
    Subjects:
      – SubjectFull: Test Construction
        Type: general
      – SubjectFull: Multiple Choice Tests
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Classification
        Type: general
      – SubjectFull: Thinking Skills
        Type: general
    Titles:
      – TitleFull: ReQUESTA: A Hybrid Agentic Framework for Generating Cognitively Diverse Multiple-Choice Questions
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Yu Tian
      – PersonEntity:
          Name:
            NameFull: Shubham Chakraborty
      – PersonEntity:
          Name:
            NameFull: Linh Huynh
      – PersonEntity:
          Name:
            NameFull: Katerina Christhilf
      – PersonEntity:
          Name:
            NameFull: Micah Watanabe
      – PersonEntity:
          Name:
            NameFull: Danielle S. McNamara
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2026
          Titles:
            – TitleFull: Grantee Submission
              Type: main
ResultId 1