Analyzing the Text Contents Produced by ChatGPT: Prompts, Feature-Components in Responses, and a Predictive Model.

Saved in:

Bibliographic Details
Title:	Analyzing the Text Contents Produced by ChatGPT: Prompts, Feature-Components in Responses, and a Predictive Model.
Authors:	Leping Liu¹ liu@unr.edu
Source:	Journal of Educational Technology Development & Exchange. 2023, Vol. 16 Issue 1, p49-70. 22p.
Subject Terms:	ChatGPT, Language models, Prediction models, Logistic regression analysis, Research questions
Abstract:	ChatGPT is a large language model that uses deep learning to produce natural language and generate intelligent and relevant responses to user prompts. It comes to the field of education as an inevitable wave. Educators have to deal with it and figure out appropriate ways to use it and produce positive learning. This study explores the use of ChatGPT from the perspective of front-end users, focusing on the text-content that ChatGPT can produce for learners to learn new knowledge (e.g., a concept, a theory, or an application). The sample of this study consists of 253 ChatGPT text responses derived from three types of initial prompts/questions: general questions, specific questions, and questions with interactive prompts. Six feature components of text-information that can help learners to understand new knowledge are analyzed (concept and definition, procedure, example, comparison or contrast, deductive or inductive argument, summary). The results from Chi-square tests indicate that the presence of each feature component in the responses differs by the types of prompts. The results from a logistic regression analysis reveal that the presence of five (out of the six) feature components are significant to the probability that a response provides accurate and reliable information. The integration of using ChatGPT into learning is discussed. Further research questions are suggested. [ABSTRACT FROM AUTHOR]
	Copyright of Journal of Educational Technology Development & Exchange is the property of Society of International Chinese in Educational Technology (SICET) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Education Research Complete

Description
Abstract:	ChatGPT is a large language model that uses deep learning to produce natural language and generate intelligent and relevant responses to user prompts. It comes to the field of education as an inevitable wave. Educators have to deal with it and figure out appropriate ways to use it and produce positive learning. This study explores the use of ChatGPT from the perspective of front-end users, focusing on the text-content that ChatGPT can produce for learners to learn new knowledge (e.g., a concept, a theory, or an application). The sample of this study consists of 253 ChatGPT text responses derived from three types of initial prompts/questions: general questions, specific questions, and questions with interactive prompts. Six feature components of text-information that can help learners to understand new knowledge are analyzed (concept and definition, procedure, example, comparison or contrast, deductive or inductive argument, summary). The results from Chi-square tests indicate that the presence of each feature component in the responses differs by the types of prompts. The results from a logistic regression analysis reveal that the presence of five (out of the six) feature components are significant to the probability that a response provides accurate and reliable information. The integration of using ChatGPT into learning is discussed. Further research questions are suggested. [ABSTRACT FROM AUTHOR]
ISSN:	19418027
DOI:	10.18785/jetde.1601.03