Automatic generation of introductory programming exercises with large language models.
Saved in:
| Title: | Automatic generation of introductory programming exercises with large language models. |
|---|---|
| Authors: | Duong Ta, Nguyen Binh1 donta@smu.edu.sg, Phuc Nguyen, Hua Gia1, Gottipati, Swapna1 |
| Source: | Research & Practice in Technology Enhanced Learning. 2026, Vol. 21, p1-35. 35p. |
| Subject Terms: | *Computer programming education, *Intelligent tutoring systems, *Instructional systems, Code generators, Language models |
| Abstract: | Despite recent advances in code generation made possible by large language models (LLMs), programming is still an essential skill that computing students need to master now and in the foreseeable future. In learning programming, frequent practices with exercises set at an appropriate difficulty and knowledge level is of crucial importance for students. However, it's not a trivial task for instructors to create many good quality exercises customized for each student. Programming problems found on Internet sources such as LeetCode are mostly too challenging for novice programmers with no prior coding knowledge. Recent work in AI-enabled education has been leveraging LLMs for adaptive feedback generation on code submitted by students. Not much work has been done in generating customized exercises for students to have more practice. In this work, we propose ExGen, an automatic exercise generation system which uses LLMs such as OpenAI's GPT models to generate on-demand, customized, and ready-to-use programming exercises for individual students. ExGen is designed as a plugin to Visual Studio Code. It incorporates a set of prompting strategies for candidate exercise generation, and a novel chain of automatic filtering mechanisms to select ready-touse exercises. ExGen is convenient to use as compared to chatbots such as ChatGPT. We have conducted an extensive performance evaluation using more than 1400 generated Python exercises. We considered several prompting strategies with various keyword and seed exercise types, filtering techniques, difficulty levels, and LLMs with different generative performance and cost. The results demonstrated the effectiveness of ExGen's design and implementation. [ABSTRACT FROM AUTHOR] |
| Copyright of Research & Practice in Technology Enhanced Learning is the property of Asia-Pacific Society for Computers in Education (APSCE) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
Be the first to leave a comment!
