Automatic generation of introductory programming exercises with large language models.
Saved in:
| Title: | Automatic generation of introductory programming exercises with large language models. |
|---|---|
| Authors: | Duong Ta, Nguyen Binh1 donta@smu.edu.sg, Phuc Nguyen, Hua Gia1, Gottipati, Swapna1 |
| Source: | Research & Practice in Technology Enhanced Learning. 2026, Vol. 21, p1-35. 35p. |
| Subject Terms: | *Computer programming education, *Intelligent tutoring systems, *Instructional systems, Code generators, Language models |
| Abstract: | Despite recent advances in code generation made possible by large language models (LLMs), programming is still an essential skill that computing students need to master now and in the foreseeable future. In learning programming, frequent practices with exercises set at an appropriate difficulty and knowledge level is of crucial importance for students. However, it's not a trivial task for instructors to create many good quality exercises customized for each student. Programming problems found on Internet sources such as LeetCode are mostly too challenging for novice programmers with no prior coding knowledge. Recent work in AI-enabled education has been leveraging LLMs for adaptive feedback generation on code submitted by students. Not much work has been done in generating customized exercises for students to have more practice. In this work, we propose ExGen, an automatic exercise generation system which uses LLMs such as OpenAI's GPT models to generate on-demand, customized, and ready-to-use programming exercises for individual students. ExGen is designed as a plugin to Visual Studio Code. It incorporates a set of prompting strategies for candidate exercise generation, and a novel chain of automatic filtering mechanisms to select ready-touse exercises. ExGen is convenient to use as compared to chatbots such as ChatGPT. We have conducted an extensive performance evaluation using more than 1400 generated Python exercises. We considered several prompting strategies with various keyword and seed exercise types, filtering techniques, difficulty levels, and LLMs with different generative performance and cost. The results demonstrated the effectiveness of ExGen's design and implementation. [ABSTRACT FROM AUTHOR] |
| Copyright of Research & Practice in Technology Enhanced Learning is the property of Asia-Pacific Society for Computers in Education (APSCE) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 192754603 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Automatic generation of introductory programming exercises with large language models. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Duong+Ta%2C+Nguyen+Binh%22">Duong Ta, Nguyen Binh</searchLink><relatesTo>1</relatesTo><i> donta@smu.edu.sg</i><br /><searchLink fieldCode="AR" term="%22Phuc+Nguyen%2C+Hua+Gia%22">Phuc Nguyen, Hua Gia</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Gottipati%2C+Swapna%22">Gottipati, Swapna</searchLink><relatesTo>1</relatesTo> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Research+%26+Practice+in+Technology+Enhanced+Learning%22">Research & Practice in Technology Enhanced Learning</searchLink>. 2026, Vol. 21, p1-35. 35p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Computer+programming+education%22">Computer programming education</searchLink><br />*<searchLink fieldCode="DE" term="%22Intelligent+tutoring+systems%22">Intelligent tutoring systems</searchLink><br />*<searchLink fieldCode="DE" term="%22Instructional+systems%22">Instructional systems</searchLink><br /><searchLink fieldCode="DE" term="%22Code+generators%22">Code generators</searchLink><br /><searchLink fieldCode="DE" term="%22Language+models%22">Language models</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Despite recent advances in code generation made possible by large language models (LLMs), programming is still an essential skill that computing students need to master now and in the foreseeable future. In learning programming, frequent practices with exercises set at an appropriate difficulty and knowledge level is of crucial importance for students. However, it's not a trivial task for instructors to create many good quality exercises customized for each student. Programming problems found on Internet sources such as LeetCode are mostly too challenging for novice programmers with no prior coding knowledge. Recent work in AI-enabled education has been leveraging LLMs for adaptive feedback generation on code submitted by students. Not much work has been done in generating customized exercises for students to have more practice. In this work, we propose ExGen, an automatic exercise generation system which uses LLMs such as OpenAI's GPT models to generate on-demand, customized, and ready-to-use programming exercises for individual students. ExGen is designed as a plugin to Visual Studio Code. It incorporates a set of prompting strategies for candidate exercise generation, and a novel chain of automatic filtering mechanisms to select ready-touse exercises. ExGen is convenient to use as compared to chatbots such as ChatGPT. We have conducted an extensive performance evaluation using more than 1400 generated Python exercises. We considered several prompting strategies with various keyword and seed exercise types, filtering techniques, difficulty levels, and LLMs with different generative performance and cost. The results demonstrated the effectiveness of ExGen's design and implementation. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Research & Practice in Technology Enhanced Learning is the property of Asia-Pacific Society for Computers in Education (APSCE) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=192754603 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.58459/rptel.2026.21025 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 35 StartPage: 1 Subjects: – SubjectFull: Computer programming education Type: general – SubjectFull: Intelligent tutoring systems Type: general – SubjectFull: Instructional systems Type: general – SubjectFull: Code generators Type: general – SubjectFull: Language models Type: general Titles: – TitleFull: Automatic generation of introductory programming exercises with large language models. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Duong Ta, Nguyen Binh – PersonEntity: Name: NameFull: Phuc Nguyen, Hua Gia – PersonEntity: Name: NameFull: Gottipati, Swapna IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 01 Text: 2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 17932068 Numbering: – Type: volume Value: 21 Titles: – TitleFull: Research & Practice in Technology Enhanced Learning Type: main |
| ResultId | 1 |
