View in EDS

Learning with large language models: beyond prompt engineering.

Saved in:
Bibliographic Details
Title: Learning with large language models: beyond prompt engineering.
Authors: Ding, Lu1 (AUTHOR) luding@southalabama.edu, O'Berry, Robert2 (AUTHOR) oberry@southalabama.edu, Tallent, Hunter2 (AUTHOR) htallent@southalabama.edu, Chhetri G C, Sanju Gharti1 (AUTHOR) sg2134@jagmail.southalabama.edu, Gapud, Albert3 (AUTHOR) gapud@southalabama.edu
Source: Education & Information Technologies. Jun2026, Vol. 31 Issue 9, p2877-2901. 25p.
Subject Terms: *Student engagement, *Machine learning, *Science education, Prompt engineering, ChatGPT, Language models
Abstract: With the advancement and increased accessibility of artificial intelligence, particularly large language models (LLMs), LLMs have garnered attention in education and are postulated to hold great promise in supporting learning. Nevertheless, these promises come with many challenges. The foremost challenge is the lack of awareness of LLMs' potential and the misuse of these tools. Researchers and educators have proposed frameworks and conducted studies to address these challenges, such as teaching prompt engineering (PE) to students to enhance their interactions with LLMs. However, we take a step further and propose the HIIC model (Human to Initiate, Interact, and Critique), arguing that learning with LLMs requires not only PE techniques but also the ability to ask additional questions and make warranted judgments about LLM-generated responses. In this study, students enrolled in a physics class were instructed to converse with ChatGPT (an LLM) about their incorrect answers on an exam. We utilized the ICAP framework (Interactive, Constructive, Active, and Passive) to analyze the additional questions students asked ChatGPT and examined the strategies they used to evaluate ChatGPT's responses. The results showed that students had limited engagement with ChatGPT, meaning they either did not ask any additional questions or asked only surface-level ones, such as fact-checking. Moreover, students relied heavily on anecdotal experiences or provided no reference when evaluating ChatGPT's responses. However, students who were more cognitively engaged with prompting showed higher learning gains than those who only superficially engaged in prompting questions. These findings highlight the need for student training that goes beyond teaching PE to focus on interactive engagement with LLMs, rather than using LLMs merely to complete tasks (which may contribute to plagiarism concerns). The implications and limitations of the study are also discussed at the end. [ABSTRACT FROM AUTHOR]
Copyright of Education & Information Technologies is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
Description
Abstract:With the advancement and increased accessibility of artificial intelligence, particularly large language models (LLMs), LLMs have garnered attention in education and are postulated to hold great promise in supporting learning. Nevertheless, these promises come with many challenges. The foremost challenge is the lack of awareness of LLMs' potential and the misuse of these tools. Researchers and educators have proposed frameworks and conducted studies to address these challenges, such as teaching prompt engineering (PE) to students to enhance their interactions with LLMs. However, we take a step further and propose the HIIC model (Human to Initiate, Interact, and Critique), arguing that learning with LLMs requires not only PE techniques but also the ability to ask additional questions and make warranted judgments about LLM-generated responses. In this study, students enrolled in a physics class were instructed to converse with ChatGPT (an LLM) about their incorrect answers on an exam. We utilized the ICAP framework (Interactive, Constructive, Active, and Passive) to analyze the additional questions students asked ChatGPT and examined the strategies they used to evaluate ChatGPT's responses. The results showed that students had limited engagement with ChatGPT, meaning they either did not ask any additional questions or asked only surface-level ones, such as fact-checking. Moreover, students relied heavily on anecdotal experiences or provided no reference when evaluating ChatGPT's responses. However, students who were more cognitively engaged with prompting showed higher learning gains than those who only superficially engaged in prompting questions. These findings highlight the need for student training that goes beyond teaching PE to focus on interactive engagement with LLMs, rather than using LLMs merely to complete tasks (which may contribute to plagiarism concerns). The implications and limitations of the study are also discussed at the end. [ABSTRACT FROM AUTHOR]
ISSN:13602357
DOI:10.1007/s10639-026-13924-2