View in EDS HTML Full Text PDF Full Text

ChatGPT-Produced Content as a Resource in the Language Education Classroom: A Guiding Hand

Saved in:
Bibliographic Details
Title: ChatGPT-Produced Content as a Resource in the Language Education Classroom: A Guiding Hand
Language: English
Authors: Rod E. Case (ORCID 0000-0003-1175-648X), Leping Liu (ORCID 0000-0001-5859-8189)
Source: Computers in the Schools. 2025 42(2):187-211.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 25
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Artificial Intelligence, Natural Language Processing, Preservice Teachers, Writing Evaluation, Technology Uses in Education, Language Teachers, State Universities, Writing (Composition), Preservice Teacher Education, Teacher Education Programs, Student Attitudes
DOI: 10.1080/07380569.2024.2442700
ISSN: 0738-0569
1528-7033
Abstract: While there is much in teacher education which examines the perceptions of preservice teacher to the recent introduction of ChatGPT. This study examines preservice teachers' appraisals of ChatGPT-produced text. Using a quasi-experimental within-­subject design with 30 participants, Chi-Square tests examined appraisals under three conditions. Results indicated that appraisal of ChatGPT-produced text was higher when prepared by the instructor among students who demonstrated an understanding of the pedagogical content versus those who did not. Appraisals were also sensitive to where ChatGPT-produced text was placed in the assignment, favoring use of ChatGPT at beginning of the assignment versus in the middle. A discussion of implications for using ChatGPT-produced text is included.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1493362
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFvQp5KLmpUEwoO6_G1fKpkAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDM0rv9Fjpyj4ty3_UQIBEICBmqPjD4Y-SpeZcVaR6ewWsfYp4Igep20aehKlP_d_ni7alKcdvXvZTnPiAE4-6ghc4rDD5HQ4RB158kBFS3Ws4g8SLFG3KlQ0CzOFNKXwrQtb6mRJ7kPbObVmWTbJc3YGXOKlBhOANEQc-z6XbQWh2f-hKtIYYqXQdh9fWCWfmTdSlQ5Yi3qpA2Hk1AMBQCk5MHmGEZ9-_aBK5ME=
Text:
  Availability: 1
  Value: <anid>AN0185256566;cit01apr.25;2025May22.01:35;v2.2.500</anid> <title id="AN0185256566-1">ChatGPT-Produced Content as a Resource in the Language Education Classroom: A Guiding Hand </title> <p>While there is much in teacher education which examines the perceptions of preservice teacher to the recent introduction of ChatGPT. This study examines preservice teachers' appraisals of ChatGPT-produced text. Using a quasi-experimental within--subject design with 30 participants, Chi-Square tests examined appraisals under three conditions. Results indicated that appraisal of ChatGPT-produced text was higher when prepared by the instructor among students who demonstrated an understanding of the pedagogical content versus those who did not. Appraisals were also sensitive to where ChatGPT-produced text was placed in the assignment, favoring use of ChatGPT at beginning of the assignment versus in the middle. A discussion of implications for using ChatGPT-produced text is included.</p> <p>Keywords: Language teacher education; ChatGPT; Sociocultural theory</p> <hd id="AN0185256566-2">Introduction</hd> <p>Since the introduction of ChatGPT, a chatbot that uses artificial intelligence (AI), educators and the larger public sector have both praised and questioned its functionality (e.g., Javier & Moorhouse, [<reflink idref="bib16" id="ref1">16</reflink>]; Moorhouse, [<reflink idref="bib28" id="ref2">28</reflink>]; Moorhouse & Kohnke, [<reflink idref="bib29" id="ref3">29</reflink>]; Yang & Appleget, [<reflink idref="bib42" id="ref4">42</reflink>]). Within teacher education research, ChatGPT is lauded for its ability to quickly produce lessons (e.g., Farrokhnia et al., [<reflink idref="bib11" id="ref5">11</reflink>]), classroom exercises, and assessments (Li et al., [<reflink idref="bib25" id="ref6">25</reflink>]; van den Berg & Du Plessis, [<reflink idref="bib38" id="ref7">38</reflink>]). Further, its use with preservice teachers is linked to gains in critical thinking skills (Li et al., [<reflink idref="bib25" id="ref8">25</reflink>]). Enthusiasm for these early findings into its potential as an instructional tool has inspired others to provide highly detailed descriptions of ChatGPT-generated lesson plans, materials and assessments that serve as instructional resources for teachers (e.g., Kohnke et al., [<reflink idref="bib21" id="ref9">21</reflink>]; Kostka & Toncelli, [<reflink idref="bib22" id="ref10">22</reflink>]).</p> <p>Beyond education, ChatGPT has demonstrated its prowess across multiple fields.Researchers have confirmed ChatGPT is capable of analyzing complex texts, creating sound arguments, and passing exams across disparate fields such as medicine (Gilson et al., [<reflink idref="bib12" id="ref11">12</reflink>]; Huh, [<reflink idref="bib15" id="ref12">15</reflink>]), physics/engineering (Yeadon et al., [<reflink idref="bib43" id="ref13">43</reflink>]), and law (Katz et al., [<reflink idref="bib19" id="ref14">19</reflink>]). In the fields of literature and philosophy, Elkins and Chun ([<reflink idref="bib10" id="ref15">10</reflink>]) note that it provides insightful and nuanced arguments by relying on a strong command of form and rhetoric. However, they also argue that is also lacks a human-driven type of common sense and conclude that while its abilities are impressive it falls short of human writers.</p> <p>Despite these advances, teacher educators have not explored the potential of ChatGPT-produced texts in the classroom. Early evidence studies focused on preservice teachers' perceptions of ChatGPT (e.g., Javier & Moorhouse, [<reflink idref="bib16" id="ref16">16</reflink>]; Moorhouse, [<reflink idref="bib28" id="ref17">28</reflink>]; Moorhouse & Kohnke, [<reflink idref="bib29" id="ref18">29</reflink>]; Yang & Appleget, [<reflink idref="bib42" id="ref19">42</reflink>]) or its potential risks, such as facilitating academic plagiarism due to its ability to produce text indistinguishable from human writing (Khalaf, [<reflink idref="bib20" id="ref20">20</reflink>]). However, less is known about how ChatGPT-produced text can support pedagogical content learning in teacher education that activities which incorporate the use of ChatGPT build a deeper understanding of course content among preservice teachers than activities which do not, particularly when teacher educators structure the prompts given ChatGPT.</p> <p>Emerging evidence (Author & Author, [<reflink idref="bib2" id="ref21">2</reflink>], under review) suggests that incorporating ChatGPT into instructional tasks can enhance preservice teachers' understanding of pedagogical content. Author & Author, [<reflink idref="bib2" id="ref22">2</reflink>] (under review) showed that students benefited more when they critiqued a ChatGPT-produced lesson prepared by the teacher, versus prompts that they created themselves. Furthermore, ChatGPT-produced text was most effective when introduced at the beginning of the instructional task and prompted by the instructor, rather than as a follow-up and prompted by the student. The authors concluded that this was largely due to the fact that the instructor had a deeper knowledge of the course content than the students, allowing him to prompt ChatGPT with more knowledgeable prompts and produce higher quality responses. The researchers called for greater research as to why placing it at the initial position in the task was significant, confirming research which suggests that the quality of the prompt, which is dependent upon an understanding of how ChatGPT constructs responses and knowledge of the content area, is a strong predictor of the accuracy of the response (Gilson et al., [<reflink idref="bib12" id="ref23">12</reflink>]</p> <p>This research, which was conducted in 2024, builds on work by Author and Author ([<reflink idref="bib2" id="ref24">2</reflink>]) (under review). The purpose of this study is to explore how language teacher educators use ChatGPT-produced texts as a resource to explore pedagogical content without ignoring questions about its ability to provide insights into a field that is driven by human experience. To do this, a content analysis, a research method which involves the coding of texts into the themes or, in this case, concepts, in order to conduct a quantitative analysis (Weber, [<reflink idref="bib40" id="ref25">40</reflink>]), of students' work was completed over the course of two semesters. The analysis examined the extent to which preservice teachers' appraisals of ChatGPT-produced text were associated with their understanding of the course content, positioning the use of ChatGPT-produced text at the beginning and the middle of the instructional sequences.</p> <hd id="AN0185256566-3">Literature review</hd> <p></p> <hd id="AN0185256566-4">ChatGPT in language teacher education</hd> <p>Work in language teacher education (LTE) has focused on linking perceptions of ChatGPT to preservice teachers' willingness to use it in instructional practice. While limited in their scope, these studies are often classroom-based accounts that identify the extent to which the preservice teachers' attitudes about ChatGPT, experience with using it, and knowledge of its functions are reliable indicators of their likelihood of using it (e.g., Javier & Moorhouse, [<reflink idref="bib16" id="ref26">16</reflink>]; Moorhouse, [<reflink idref="bib28" id="ref27">28</reflink>]; Moorhouse & Kohnke, [<reflink idref="bib29" id="ref28">29</reflink>]; Yang & Appleget, [<reflink idref="bib42" id="ref29">42</reflink>]). Giving preservice teachers the opportunity to use ChatGPT to plan instruction and also guide students as they use ChatGPT in the classroom is key, but, while it is tempting to assume that preservice teachers already have detailed knowledge of ChatGPT, many do not. Preservice teachers who have not had the opportunity to use it at home, may not feel as confident in using it to create lessons or classroom materials as those who have (Moorhouse, [<reflink idref="bib28" id="ref30">28</reflink>]).</p> <p>The benefits of preservice teachers using ChatGPT are beginning to emerge. While many have documented how it frees teachers from routine tasks such as creating lessons, worksheets, and assessments (e.g., Li et al., [<reflink idref="bib25" id="ref31">25</reflink>]; van den Berg & Du Plessis, [<reflink idref="bib38" id="ref32">38</reflink>]), more interesting is research that suggests that it can enhance critical thinking skills (Li et al., [<reflink idref="bib25" id="ref33">25</reflink>]; van den Berg & Du Plessis, [<reflink idref="bib38" id="ref34">38</reflink>]). Li et al. ([<reflink idref="bib25" id="ref35">25</reflink>]), for instance, found that preservice teachers who had interacted with ChatGPT, labeled as the "Human-Machine" group, demonstrated higher levels of critical thinking on post-test measures than preservice teachers who had only interacted with the instructor, labeled the "Human-Human" group. Li et al. ([<reflink idref="bib25" id="ref36">25</reflink>]) drew on work by Rusandi et al. ([<reflink idref="bib33" id="ref37">33</reflink>]) and Ciechanowski et al. ([<reflink idref="bib5" id="ref38">5</reflink>]) to argue that the reason for the increase in critical thinking might be attributable to the fact that preservice teachers often checked ChatGPT's work against outside sources.</p> <p>Still, allowing students to use ChatGPT in the classroom without guidance is not recommended. Building on specific work by Li et al. ([<reflink idref="bib25" id="ref39">25</reflink>]) and others more broadly (e.g., Cooper, [<reflink idref="bib8" id="ref40">8</reflink>]; Farrokhnia et al., [<reflink idref="bib11" id="ref41">11</reflink>]; Skrabut, [<reflink idref="bib36" id="ref42">36</reflink>]; van den Berg & Du Plessis, [<reflink idref="bib38" id="ref43">38</reflink>]), Jeon and Lee ([<reflink idref="bib17" id="ref44">17</reflink>]) found that few teachers advocated the practice of allowing students to experiment with ChatGPT without close supervision. Most teachers called for a mix of using ChatGPT to assist in lesson planning and then closely supervising students during its use. While helpful, these researchers have not examined how preservice teachers learn to navigate the affordances and constraints of ChatGPT-produced lessons, assessment and content while they are still learning to teach.</p> <hd id="AN0185256566-5">The affordances and constraints of ChatGPT-produced text</hd> <p>To date, research into the capabilities of ChatGPT to produce academic texts or classroom materials appropriate for the preservice teachers pursuing teacher licensure has been limited to explorations of preservice or inservice teachers' perceptions of ChatGPT's instructional prowess (e.g., Javier & Moorhouse, [<reflink idref="bib16" id="ref45">16</reflink>]; Moorhouse, [<reflink idref="bib28" id="ref46">28</reflink>]; Moorhouse & Kohnke, [<reflink idref="bib29" id="ref47">29</reflink>]; Yang & Appleget, [<reflink idref="bib42" id="ref48">42</reflink>]). This scholarship often connects teachers' perceptions of ChatGPT to their predictions of how ChatGPT might perform in the classroom. While valuable, it often lacks an exploration of the nature and quality of ChatGPT and is completed with a limited number of examples that were not subjected to experimental conditions.</p> <p>In response to this gap in the research, the review below details existing research into the nature of ChatGPT-produced texts. While the research is drawn from fields outside of teacher education (e.g., medicine, Gilson et al. ([<reflink idref="bib12" id="ref49">12</reflink>]), Huh ([<reflink idref="bib15" id="ref50">15</reflink>]), physics/engineering, Yeadon et al. ([<reflink idref="bib43" id="ref51">43</reflink>]), literature/philosophy, Elkins and Chun ([<reflink idref="bib10" id="ref52">10</reflink>]) and law Katz et al. ([<reflink idref="bib19" id="ref53">19</reflink>]), there is much that can be inferred in building an introductory understanding of ChatGPT-produced text for the teacher educator.</p> <p>One strand of this research explores ChatGPT's performance on various standardized exams as an indicator of its capability to process content accurately. While Huh ([<reflink idref="bib15" id="ref54">15</reflink>]) found that ChatGPT did not outperform humans on a standardized test of parasitology that included both multiple choice and written responses, Huh ([<reflink idref="bib15" id="ref55">15</reflink>]) also noted that one possible explanation for the lower scores might be that ChatGPT's inability to interpret tables and images competently. Gilson et al. ([<reflink idref="bib12" id="ref56">12</reflink>]) accounted for these limitations in a measure of ChatGPT's medical knowledge as demonstrated on its performance on the United States Medical Exam. Questions that contained images or tables were removed, and sample questions in each prompt were added. Results showed performance on the first level of the exam to be comparable to third-year medical students but declining performance on the remaining two levels. ChatGPT-produced writing samples and evidenced highly logical explanations throughout.</p> <p>Other research corroborates work by Huh ([<reflink idref="bib15" id="ref57">15</reflink>]) and Gilson et al. ([<reflink idref="bib12" id="ref58">12</reflink>]), confirming ChatGPT's capabilities to integrate complex content into logical well-written text (e.g., Katz et al., [<reflink idref="bib19" id="ref59">19</reflink>]; Yeadon et al., [<reflink idref="bib43" id="ref60">43</reflink>]; Zhai, [<reflink idref="bib44" id="ref61">44</reflink>]). Findings from Katz et al. ([<reflink idref="bib19" id="ref62">19</reflink>]), for instance, measured performance on the Universal Bar Examination (UBE) and its three components: (<reflink idref="bib1" id="ref63">1</reflink>) the multistate bar exam (MBE), multistate essay exam (MEE), and multistate performance test (MPT). ChatGPT-4 outperformed the pass rate of law students on the MBE, but, more specific to the question of its capabilities to create documents which demonstrate high levels of analytical thought, ChatGPT-4 outperformed previous iterations of ChatGPT on the MEE. Its essays demonstrated the ability to analyze complex legal issues which require the writer to deduce answers through the use of entailment and competently connect knowledge of the law to the issue of interest.</p> <p>In a turn from exploring the capabilities of ChatGPT on standardized tests in the sciences, Elkins and Chun ([<reflink idref="bib10" id="ref64">10</reflink>]) delved into ChatGPT's performance on examining literature and philosophy. Drawing on their experiences as professors of literature and philosophy, they examined ChatGPT's ability to form arguments, Elkins and Chun ([<reflink idref="bib10" id="ref65">10</reflink>]) found that while ChatGPT described the insights that drive works of philosophy by Plato and Nietzsche with humor, irony and playfulness, they argued that without "judicious" selection of its work ChatGPT would probably not pass a writer's Turing Test. In other words, it could not produce a text that is indistinguishable from one created by a human. Its writing is inconsistent. While it readily emulates rhetorical form and explore complex philosophical arguments, it also lacks what they described as "commonsense and foundational knowledge" (p. 12) which is typically present in texts created by humans. While findings from Elkins and Chun ([<reflink idref="bib10" id="ref66">10</reflink>]) stop short of proposing that ChatGPT could emulate human writing, they leave open the possibility for exploration of its potential as interesting source of writing and knowledge which students could learn from.</p> <hd id="AN0185256566-6">Theoretical framework</hd> <p>According to Shabani ([<reflink idref="bib35" id="ref67">35</reflink>]), there is common agreement that three themes drive Vygotsky ([<reflink idref="bib39" id="ref68">39</reflink>]) work: (a) a close examination of individual development to understand intellectual growth, (b) the claim that cognitive growth is rooted in social experience, and (c) a focus on the mediated nature of learning. Of the three, the third is most closely related to teacher education, as it speaks to the central role of mediation and the Zone of Proximal Development (ZPD) in classroom instruction. Johnson and Golombek ([<reflink idref="bib18" id="ref69">18</reflink>]) sociocultural framework builds on Vygotsky's ([<reflink idref="bib39" id="ref70">39</reflink>]) work and details eight propositions intended to guide language teacher educators in their instruction of preservice teachers. Three concepts central to Johnson and Golombek ([<reflink idref="bib18" id="ref71">18</reflink>]) framework, mediation, everyday concept and academic concept, are examined below.</p> <p>Mediation is the driving force for all instruction and highlights the central role that the teacher educator plays as the more knowledgeable other in interactions with preservice teachers. The teacher educator is tasked with creating mediational spaces for preservice teachers in which they are encouraged to "play/step" into becoming teachers. Johnson and Golombek ([<reflink idref="bib18" id="ref72">18</reflink>]) call on teacher educators to establish these spaces as "safe-zones" for trying out new ideas that their preservice teachers are not yet ready to carry out in the classroom without assistance. Ideally, preservice teachers should feel free to reflect on and critique their still-emerging ideas about pedagogical content. Course curriculum is structured around goal-directed activities which can include practicum, written essays, lesson planning or various kinds of discussion activities (e.g., Author, [<reflink idref="bib1" id="ref73">1</reflink>]).</p> <p>Next, learning for preservice teachers begins as an everyday concept. Everyday concepts emerge from experiences both within and outside of school settings. While preservice teachers' ideas about language, language teaching, and technology, which form from these experiences are often flawed, superficial, or unanalyzed, they are not to be dismissed or regarded as a deficit (Johnson & Golombek, [<reflink idref="bib18" id="ref74">18</reflink>]). Rather, they are the building blocks for movement toward an academic concept.</p> <p>Johnson and Golombek ([<reflink idref="bib18" id="ref75">18</reflink>]) described academic concepts as representing "systematic and generalizable knowledge of entities and phenomenon in the world" (p. 122). Unlike everyday concepts, academic concepts are systematically formed and emerge from the kinds of structured learning experiences emerge with formal instruction (Lantolf & Minakova, [<reflink idref="bib24" id="ref76">24</reflink>]). Preservice teachers who have knowledge of pedagogical content as an academic concept are able to create lessons and make judgements about their teaching that can be generalized across different contexts.</p> <hd id="AN0185256566-7">Need for research</hd> <p>How can teacher education research into the use of ChatGPT and the broader multidisciplinary exploration of the nature of ChatGPT-produced text inform examinations of ChatGPT in LTE? Teacher education research suggests that ChatGPT has the potential to increase critical thinking (Li et al., [<reflink idref="bib25" id="ref77">25</reflink>]), provided there are ample opportunities to build background experiences with using ChatGPT (Moorhouse, [<reflink idref="bib28" id="ref78">28</reflink>]). Research which explores the use of ChatGPT-produced texts in teacher education has not been forthcoming despite its potential to synthesizing large amounts of data and creating cogent, logical explanations and analyses in fields as diverse as medicine (Gilson et al., [<reflink idref="bib12" id="ref79">12</reflink>]; Huh, [<reflink idref="bib15" id="ref80">15</reflink>]), physics/engineering (Yeadon et al., [<reflink idref="bib43" id="ref81">43</reflink>]), literature/philosophy (Elkins & Chun, [<reflink idref="bib10" id="ref82">10</reflink>]) and law (Katz et al., [<reflink idref="bib19" id="ref83">19</reflink>]), suggesting that ChatGPT is able to form generalizable knowledge that can be applied to a variety of circumstances, a hallmark of an academic concept. Building on work by Author and Author ([<reflink idref="bib2" id="ref84">2</reflink>]) (under review), this study fills that gap in the research by exploring extent to which preservice teachers' appraisals of ChatGPT-produced texts were associated with the presence of academic concepts in their work.</p> <hd id="AN0185256566-8">Research questions</hd> <p>The learning outcome in this study was measured by whether or not students included academic concepts in their course work. In order to operationalize the measurement of academic concepts, two measurement criteria inferred from Johnson and Golombek ([<reflink idref="bib18" id="ref85">18</reflink>]), definition and application, were developed. The third criteria informed by research on the use of prompts in ChatGPT-produced texts (Huh, [<reflink idref="bib15" id="ref86">15</reflink>]) was technical concerns. Technical concerns were identified when students expressed concerns with the results of their prompts when using ChatGPT.</p> <p>Measurement of students' appraisals of ChatGPT-produced text were also conducted. These were sorted into three categories: rejected, ambivalent, and accepted. Research question one provided an overall measurement of the students' appraisals of their readings of ChatGPT-produced text in terms of <emph>definition</emph>, <emph>application</emph> and <emph>technical concerns</emph>. Research question two measured the extent to which students' learning outcomes from three different instructional sequences (explored more fully below) were associated with their appraisal of definition, application and technical concerns.</p> <p>The organization of the assignments varied along two elements. First, assignments differed according to whether the instructor or the students prompted ChatGPT with directions. In the second element, the placement of ChatGPT-produced text was varied. ChatGPT-produced text was introduced at one of three points in the instructional task. Each one of these variations was called an instructional sequence. In sequence one, the instructor provided the ChatGPT-produced text in the middle of the instructional task. In sequence two, the teacher educator included the ChatGPT-produced text at the beginning of the instructional task, and, finally, in sequence three, the instructor added ChatGPT-produced text at the beginning of the instructional task. Utilizing the parameters describe above, this study asked two research questions:</p> <p></p> <ulist> <item> Are students' learning outcomes from each instructional sequence associated with the overall appraisals one each type of information that they identified from the ChatGPT-produced text: (a) definition, (b) application, and (c) technical concerns?</item> <p></p> <item> Are students' learning outcomes from each instructional sequence associated with their appraisals (rejected, ambivalent, and accepted) on each type of information that they identified from ChatGPT-produced text?</item> </ulist> <hd id="AN0185256566-9">Methodology</hd> <p></p> <hd id="AN0185256566-10">Design</hd> <p>A quasi-experimental within-subject design was adopted in this study (Nicolaidou et al., [<reflink idref="bib30" id="ref87">30</reflink>]) with a nonrandomized sample of preservice teachers from an existing language teacher education class. The use of ChatGPT was integrated into the three instructional sequences with all students participating in the same course. Students' learning outcomes were examined in association with their appraisals of each type of ChatGPT-produced text that they identified during the course.</p> <p>Chi-Square (<emph>χ<sups>2</sups></emph>) tests were used for the data analysis. To determine the appropriate sample size, a priori power analysis was performed with G*Power 3.1.9.4. The <emph>α</emph> level was set at.05, and statistical power at.8 (Williamson, [<reflink idref="bib41" id="ref88">41</reflink>]). Consistent with contemporary literature (Liu, [<reflink idref="bib26" id="ref89">26</reflink>]), four effect size values were chosen: (.5), (.55), (.6), and (.65). The resulting minimum sample sizes were 39, 32, 27, and 23 for two by three (<emph>χ<sups>2</sups></emph>) tests, and 32, 26, 22, and 19 for two by two (<emph>χ<sups>2</sups></emph>) tests, respectively. The sample size of this study (<emph>N</emph> = 30) was in the appropriate range, given the expected effect size.</p> <hd id="AN0185256566-11">Participants and data sample</hd> <p>Thirty students from a western state university in the United States enrolled in a language teacher education course taught during the fall of 2023 and again in the spring of 2024 participated in this study. Ten were from the fall class, three males and seven females. Twenty from the spring class, six males and fourteen females. They were in their junior or senior years majoring English, Spanish, social studies, mathematics and science. The data sample for this study were the three written-assignments from each of the 30 participants completed in the three instructional sequences, resulting in a total of 90 assignments.</p> <hd id="AN0185256566-12">The course</hd> <p>The 16-week course was required for all juniors and seniors seeking teaching licensure. Students completed a total of seven written assignments, six discussion posts and two lesson plans. Because the course was online and asynchronous, detailed written instructions and instructor-created videos accompanied the weekly assignments. The content of the course was drawn from contemporary articles and research on language teaching and learning and introduced students to six academic concepts. The first four, funds of knowledge (de Souza, [<reflink idref="bib9" id="ref90">9</reflink>]; Moll, [<reflink idref="bib27" id="ref91">27</reflink>]), the deficit perspective (de Souza, [<reflink idref="bib9" id="ref92">9</reflink>]), Consejos (Goldsmith & Kurpius, [<reflink idref="bib13" id="ref93">13</reflink>]) and academic language (Zwiers, [<reflink idref="bib45" id="ref94">45</reflink>]), prepared students with foundational ideas about language instruction. The remaining two academic concepts included translanguaging (Solorza et al., [<reflink idref="bib37" id="ref95">37</reflink>]) and schema. While assignments varied in length and complexity, all assignments required students to demonstrate knowledge of the selected academic concept, by both expressing knowledge and use of its definition and exploring its application. This study focused on the learning activities of developing and using three academic concepts: consejos, schema, and translanguaging.</p> <hd id="AN0185256566-13">Procedures</hd> <p>Following Institutional Review Board approval, data collection and analysis began. Content coding of students' assignments was done to examine learning outcomes. ChatGPT-produced texts, prompted by either the instructor or the students themselves, were integrated into three instructional sequences across the three activities described below (Table 1). A description of each sequence follows.</p> <p>Table 1. Sequences of using ChatGPT-information in learning.</p> <p> <ephtml> <table><thead><tr><td>Learning in:</td><td>Step1</td><td>Step 2</td><td>Step 3</td><td>Step 4</td></tr></thead><tbody valign="top"><tr><td>Sequence 1</td><td>Student completes first essay on consejos</td><td><bold><italic>Instructor-prompted ChatGPT text is given to students</italic></bold></td><td>Students compare their essay from step one with ChatGPT text from step two.</td><td>Final essay formulated around the selected academic concept</td></tr><tr><td>Sequence 2</td><td><bold><italic>Student-prompted ChatGPT text</italic></bold></td><td>Student review & identify ChatGPT text</td><td>Student essay</td><td>Final essay formulated around the selected academic concept</td></tr><tr><td>Sequence 3</td><td><bold><italic>Instructor-prompted ChatGPT text</italic></bold></td><td>Student review & identify ChatGPT text</td><td>Student written essay</td><td>Final essay formulated around the selected academic concept</td></tr></tbody></table> </ephtml> </p> <p>In sequence one, the introduction of ChatGPT-produced text was prepared by the instructor and placed in the middle of the first learning activity (Table 1). This particular sequence was conducted over two weeks. In the first week, the students completed a response to Solorza's ([<reflink idref="bib37" id="ref96">37</reflink>]) description and application of consejos (Goldsmith & Kurpius, [<reflink idref="bib13" id="ref97">13</reflink>]). Two weeks later, the instructor used ChatGPT to create a response to the week-one assignment. Next, the students read ChatGPT's essay and compared it with the essay they created in week one, critiquing ChatGPT's use of consejos (Goldsmith & Kurpius, [<reflink idref="bib13" id="ref98">13</reflink>]) in terms of the two operationalized measures of academic concept, definition and application.</p> <p>Sequence two used student-prompted text and was introduced at the beginning of the instructional task. The task asked the students to use ChatGPT to create a lesson for their particular content area to apply an academic concept. They were directed to set the number of multilingual students in their class at 20%, indicate the grade level, the content area, the length of the lesson and ensure that the ChatGPT demonstrated the use of schema. Students were not given additional information such as the value of using multiple prompts or the limitations of ChatGPT to interpret tables or images. This decision provided data on their facility with ChatGPT absent formal instruction. The lesson required them to write a brief essay about the results of their efforts at creating a lesson using ChatGPT and critique its ability to explore the use of schema, the academic concept in a lesson plan.</p> <p>In sequence three, the use of ChatGPT was set at the beginning of the instructional task and instructor-prompted. Translanguaging (Solorza et al., [<reflink idref="bib37" id="ref99">37</reflink>]) was the academic concept. The instructor entered similar parameters into ChatGPT described in sequence two. As in sequence two, images and tables were avoided. Also, multiple prompts were used to ensure that the final lesson had elements of an academic concept that students would be able to identify. Finally, as with sequences one and two, students critiqued the extent to which ChatGPT's use of translanguaging reflected an academic concept.</p> <hd id="AN0185256566-14">Content coding and variables</hd> <p>The data sample consisted of 90 assignments from the 30 students. The content of each assignment was examined. Learning outcomes, students-identified ChatGPT-produced text, and students' appraisal of the ChatGPT-produced texts were coded as described below.</p> <hd id="AN0185256566-15">Learning outcome</hd> <p>To facilitate the coding of the students' work, academic concept was operationalized by focusing on the students' abilities to define and apply the selected academic concept. Based upon a close reading of Johnson and Golombek ([<reflink idref="bib18" id="ref100">18</reflink>]), an academic concept was present in a text when there was evidence of a definition of the pedagogical concept and an application. A definition required the students to "demonstrate systematic and generalizable knowledge of the academic concept" (Johnson & Golombek, [<reflink idref="bib18" id="ref101">18</reflink>], p. 122). To meet this criterion, the work should demonstrate an understanding of internal components and structure of the academic concept. For example, the assignment might include quotes, related examples which define the academic concept in the students' own words.</p> <p>The second criterion, application, required students to connect "the academic concept to personal academic activities or real-world experience"(Johnson & Golombek, [<reflink idref="bib18" id="ref102">18</reflink>], p. 122). To meet this criterion, the student should describe or interpret how effectively a lesson or instructional activity communicates the use of an academic concept. Students may relate their thinking about instructional activity from their own experience as learners, describe an experience from a practicum or classroom observation, or explore an instructional activity from the week's reading. For each assignment, content coding was conducted for the presence of definition and application. When both definition and application were present, a code of (<reflink idref="bib1" id="ref103">1</reflink>) was given; if not, a code of (0) was assigned.</p> <hd id="AN0185256566-16">Student-identified ChatGPT-produced text</hd> <p>In their assignments, the preservice teachers identified and critiqued the use of definition, application and technical concerns in ChatGPT-produced text. Definition and application are the operational terms used with academic concept. These terms are described below.</p> <p> <emph>Definition:</emph> With respect to academic concept, the students identified the definition in ChatGPT-produced texts where fundamental knowledge, developmental context, references or relevant examples were demonstrated.</p> <p>The students' writing showed an understanding of fundamental knowledge of the academic concept. This might include use of references, relevant examples and/or discussion of the definition of one of the academic concepts such as funds of knowledge (de Souza, [<reflink idref="bib9" id="ref104">9</reflink>]; Moll, [<reflink idref="bib27" id="ref105">27</reflink>]), the deficit perspective (de Souza, [<reflink idref="bib9" id="ref106">9</reflink>]), Consejos (Goldsmith & Kurpius, [<reflink idref="bib13" id="ref107">13</reflink>]) and academic language (Zwiers, [<reflink idref="bib45" id="ref108">45</reflink>]).</p> <p> <emph>Application</emph>: Students identified and critiqued ChatGPT-produced application, according to the ChatGPT descriptions of relevant or suggested, real-world or hypothetical activities and instructional ideas.</p> <p> <emph>Technical concerns:</emph> Technical concerns were identified when students expressed concerns with the results of their prompts when using ChatGPT. The most common technical concern was having to prompt ChatGPT multiple times in order to get a desired response.</p> <hd id="AN0185256566-17">Appraisal of ChatGPT-produced texts</hd> <p>The students also appraised the use of definition, application and technical concerns in ChatGPT-produced texts. The researchers coded the appraisals that the students identified. With respect to definition and application, the variable was coded into three categories: rejected, ambivalent, and accepted. These were coded as (<reflink idref="bib1" id="ref109">1</reflink>), (<reflink idref="bib2" id="ref110">2</reflink>), and (<reflink idref="bib3" id="ref111">3</reflink>), respectively:</p> <p> <emph>Rejected:</emph> The students rejected the use of ChatGPT, as they found that it did not present a knowledgeable description of the academic concept after repeated attempts at prompting it.</p> <p> <emph>Ambivalent:</emph> The students were ambivalent about the use of ChatGPT. ChatGPT represented a viable tool that could inform their practice as future teachers, but its future use is contingent upon their willingness to prompt ChatGPT several times in order to find a satisfactory response. Also, students were ambivalent when ChatGPT's discussion and definition of the required academic concept or application was not fully developed or had small errors in the response.</p> <p> <emph>Accepted</emph>: Students identified the contents as accepted when they found that all aspects of ChatGPT's response were correct and could inform their practice as future teachers without further prompting of ChatGPT.</p> <p>With respect to technical concerns, the variable was coded into two categories: concern expressed and concern not expressed, which were coded as (<reflink idref="bib1" id="ref112">1</reflink>) and (0), respectively. The association between learning outcome and the appraisals that the students identified in ChatGPT-produced text was examined.</p> <hd id="AN0185256566-18">Intra-rater reliability analysis</hd> <p>In content coding, reliability analysis is suggested to determine either the interrater reliability or the intra-rater reliability (Rourke et al., [<reflink idref="bib32" id="ref113">32</reflink>]; Weber, [<reflink idref="bib40" id="ref114">40</reflink>], p. 15). In this study, intra-rater reliability analysis was performed.</p> <p>The content coding for the four variables (academic concept, definition, application, and technical concern) in each assignment was first completed using all samples (all 90 assignments). Consistent with research from Hanselman and Author ([<reflink idref="bib14" id="ref115">14</reflink>]), the same researcher revisited the data to verify the reliability of the initial coding. As 15% of the sample was recommended as appropriate for the intra-rater reliability check (Landis & Koch, [<reflink idref="bib23" id="ref116">23</reflink>]), the researchers randomly selected and recoded 25 assignments (about 27% of the total amount of data) for the intra-rater reliability analysis. Cohen's Kappa (<emph>K</emph>) statistic was conducted to examine the level of agreement in the coding results between the two coding periods (Cohen, [<reflink idref="bib6" id="ref117">6</reflink>]). Values of Kappa between.40 and.59 are considered moderate. Between.60 and.79 are considered substantial, and above.80 are considered outstanding (Landis & Koch, [<reflink idref="bib23" id="ref118">23</reflink>]). The results of the intra-rater reliability Kappa tests on academic concept (<emph>K</emph> =.839, <emph>p</emph> <.001), definition (<emph>K</emph> =.839, <emph>p</emph> <.001), application (<emph>K</emph> =.936, <emph>p</emph> <.001), and technical concern (<emph>K</emph> =.689, <emph>p</emph> <.001) indicated that the level of agreement between the two coding periods regarding the four variables were substantial. The data from the first coding was used for data analysis.</p> <hd id="AN0185256566-19">Data analyses and results</hd> <p>A nonparametric analysis method Chi-Square test of independence was used for the data analyses of the study. The results are presented according to the research questions below.</p> <hd id="AN0185256566-20">Data analyses and results for research question one</hd> <p>In the data analysis for research question one, learning outcomes and overall appraisal from each instructional sequence was examined using Chi-Square (<emph>χ<sups>2</sups></emph>) tests. Three Chi-Square (<emph>χ<sups>2</sups></emph>) tests were conducted in each sequence: (a) learning outcome by definition, (b) learning outcome by application, and (c) learning outcome by technical concerns.</p> <p>In instructional sequence one, the learning outcome was significantly associated with the students' overall appraisal of definition in ChatGPT-produced texts (<emph>χ<sups>2</sups></emph> = 13.526, <emph>N</emph> = 28, <emph>p</emph> =.004, effect size <emph>Cramer's V =</emph>.695, <emph>p</emph> =.004) but not significantly associated with students' appraisals of either application (<emph>χ<sups>2</sups></emph> = 3.073, <emph>N</emph> = 28, <emph>p</emph> =.215, <emph>Cramer's V =</emph>.331, <emph>p</emph> =.215) or technical concern(<emph>χ<sups>2</sups></emph> = 1.163, <emph>N</emph> = 28, <emph>p</emph> =.281, <emph>Cramer's V =</emph>.204, <emph>p</emph> =.281).</p> <p>In sequence two, the learning outcome was significantly associated with the students' overall appraisal of definition in ChatGPT-produced texts (<emph>χ<sups>2</sups></emph> = 10.383, <emph>N</emph> = 27, <emph>p</emph> =.006, effect size <emph>Cramer's V =</emph>.620, <emph>p</emph> =.006), but not significantly associated with either application (<emph>χ<sups>2</sups></emph> = 1.485, <emph>N</emph> = 27, <emph>p</emph> =.476, <emph>Cramer's V =</emph>.235, <emph>p</emph> =.476) or technical concern (<emph>χ<sups>2</sups></emph> =.667, <emph>N</emph> = 27, <emph>p</emph> =.414, <emph>Cramer's V =</emph>.157, <emph>p</emph> =.414).</p> <p>In sequence three, the learning outcome was significantly associated with the students' overall appraisal of definition in ChatGPT-produced texts (<emph>χ<sups>2</sups></emph>= 10.887, <emph>N</emph> = 27, <emph>p</emph> =.012, effect size <emph>Cramer's V =</emph>.635, <emph>p</emph> =.012) and the ChatGPT-produced application (<emph>χ<sups>2</sups></emph> = 17.532, <emph>N</emph> = 27, <emph>p</emph> <.001, <emph>Cramer's V =</emph>.806, <emph>p</emph> <.001) but not significantly associated with their appraisal of technical concern (<emph>χ<sups>2</sups></emph> = 1.485, <emph>N</emph> = 27, <emph>p</emph> =.476, <emph>Cramer's V =</emph>.235, <emph>p</emph> =.476).</p> <p>In summary, the students' overall appraisal of definition in ChatGPT-produced texts was associated with the learning outcome in all three sequences. The overall appraisal of application in ChatGPT-produced texts was associated with the learning outcome only in sequence three. The overall appraisal of technical concern with respect to ChatGPT-produced texts was not associated with any of the three sequences. No further tests were performed on the appraisal of technical concern.</p> <hd id="AN0185256566-21">Data analyses and results for research question two</hd> <p>In the data analysis for research question two, learning outcomes from each sequence were examined with Chi-Square (<emph>χ<sups>2</sups></emph>) tests by the specific appraisal (rejected, ambivalent, and accepted) of definition in ChatGPT-produced texts. Second, the learning outcome from sequence three was examined with Chi-Square (<emph>χ<sups>2</sups></emph>) tests by the specific appraisal of application in ChatGPT-produced texts. The following are the results from the Chi-Square (<emph>χ<sups>2</sups></emph>) tests.</p> <hd id="AN0185256566-22">Sequence one: Learning outcomes by the appraisal of definition</hd> <p>In sequence one, the instructor-prompted ChatGPT-produced text was included in the middle of the instructional sequence. Results indicated that the proportion of correct assignments in which the students' appraisal of definition in ChatGPT-produced texts was "accepted" waslarger than the proportion of correct assignments that were "rejected" or appraised as "ambivalent" (Table 2).</p> <p>Table 2. Chi-square comparison tests for definition in sequence 1.</p> <p> <ephtml> <table><thead><tr><td>GPT info-quality as identified</td><td>Learning outcomes</td><td>Chi-square results</td></tr><tr><td>0<sup>a</sup></td><td>1<sup>b</sup></td><td><italic>χ<sup>2</sup></italic></td><td><italic>p</italic></td><td><italic>Phi (φ)</italic><xref ref-type="table-fn" rid="tfn2">*</xref></td></tr></thead><tbody valign="top"><tr><td /><td>(<italic>n</italic> = 12)</td><td>(<italic>n</italic> = 6)</td><td /><td /><td /></tr><tr><td>Rejected</td><td char=".">3 (40%)</td><td char=".">0 (0 %)</td><td char=".">1.80</td><td char=".">.180</td><td char=".">.318</td></tr><tr><td>Ambivalent</td><td char=".">9 (60%)</td><td char=".">6 (100 %)</td><td /><td /><td /></tr><tr><td /><td>(<italic>n</italic> = 3)</td><td>(<italic>n</italic> = 9)</td><td /><td /><td /></tr><tr><td>Rejected</td><td char=".">3 (100%)</td><td char=".">0 (0 %)</td><td char=".">12.00</td><td char="."><.001</td><td char=".">1.00</td></tr><tr><td>Accepted</td><td char=".">0 (0 %)</td><td char=".">9 (100%)</td><td /><td /><td /></tr><tr><td /><td>(<italic>n</italic> = 9)</td><td>(<italic>n</italic> = 15)</td><td /><td /><td /></tr><tr><td>Ambivalent</td><td char=".">9 (100%)</td><td char=".">6 (40.0%)</td><td char=".">8.64</td><td char=".">.003</td><td char=".">.600</td></tr><tr><td>Accepted</td><td char=".">0 (0 %)</td><td char=".">9 (60.0%)</td><td /><td /><td /></tr></tbody></table> </ephtml> </p> <p>1 <emph>Notes:</emph></p> <ulist> <item>2 All the <emph>Phi (φ)</emph> tests had the same significant level of <emph>p</emph> as in each of the <emph>χ<sups>2</sups></emph> tests.</item> <item>3 (1<sups>b</sups>) Assignments that the academic concepts were correctly formulated.</item> <item>4 (<sups>0a</sups>) Assignments that the academic concepts were not correctly formulated.</item> </ulist> <p>For example, the results of the third Chi-Square (<emph>χ<sups>2</sups></emph>) test were significant (<emph>χ<sups>2</sups></emph> = 8.64, <emph>N</emph> = 24, <emph>p</emph> =.003, effect size <emph>Phi (φ) =</emph>.60), indicating that the proportions of correct assignments (learning outcome =1) and incorrect assignments (learning outcome = 0) varied significantly between those in which definition was identified as "ambivalent" and as "accepted." In the 15 correct assignments, 9 (60%) were identified as "accepted" on definition, which was higher than "ambivalent" 6 (40%). In the 9 incorrect assignments, students did not identify any one (0%) as "accepted" but 9 (100%) identified definition as "ambivalent."</p> <hd id="AN0185256566-23">Sequence two: Learning outcomes by the appraisal of definition</hd> <p>In sequence two, the student-prompted ChatGPT-produced text was provided at the beginning of the instructional sequence. Results indicated that the proportion of correct assignments in which the students' appraisal of definition in ChatGPT-produced texts was "accepted" was smaller than "rejected" or appraised as "ambivalent" (Table 3).</p> <p>Table 3. Chi-square comparison tests for definition in sequence 2.</p> <p> <ephtml> <table><thead><tr><td>GPT info-quality as identified</td><td>Learning outcomes</td><td>Chi-square results</td></tr><tr><td>0<sup>a</sup></td><td>1<sup>b</sup></td><td><italic>χ<sup>2</sup></italic></td><td><italic>p</italic></td><td><italic>Phi (φ)</italic><xref ref-type="table-fn" rid="tfn6">*</xref></td></tr></thead><tbody valign="top"><tr><td /><td>(<italic>n</italic> = 4)</td><td>(<italic>n</italic> = 6)</td><td /><td /><td /></tr><tr><td>Rejected</td><td char=".">1 (25%)</td><td char=".">3 (50%)</td><td char=".">0.625</td><td char=".">.429</td><td char=".">.250</td></tr><tr><td>Ambivalent</td><td char=".">3 (75%)</td><td char=".">3 (50%)</td><td /><td /><td /></tr><tr><td /><td>(<italic>n</italic> = 17)</td><td>(<italic>n</italic> = 4)</td><td /><td /><td /></tr><tr><td>Rejected</td><td char=".">1 (5.9%)</td><td char=".">3 (75%)</td><td char=".">10.032</td><td char=".">.002</td><td char=".">.691</td></tr><tr><td>Accepted</td><td char=".">16 (94.1%)</td><td char=".">1 (25%)</td><td /><td /><td /></tr><tr><td /><td>(<italic>n</italic> = 19)</td><td>(<italic>n</italic> = 4)</td><td /><td /><td /></tr><tr><td>Ambivalent</td><td char=".">3 (40%)</td><td char=".">3 (75%)</td><td char=".">6.008</td><td char=".">.014</td><td char=".">.511</td></tr><tr><td>Accepted</td><td char=".">16 (60%)</td><td char=".">1 (25%)</td><td /><td /><td /></tr></tbody></table> </ephtml> </p> <ulist> <item>5 <emph>Notes:</emph></item> <item>6 All the <emph>Phi (φ)</emph> tests had the same significant level of <emph>p</emph> as in each of the <emph>χ<sups>2</sups></emph> tests.</item> <item>7 (<sups>1b</sups>) Assignments that the academic concepts were correctly formulated.</item> <item>8 (<sups>0a</sups>) Assignments that the academic concepts were not correctly formulated.</item> </ulist> <p>For example, the results of the third Chi-Square (<emph>χ<sups>2</sups></emph>) test were significant (<emph>χ<sups>2</sups></emph> = 6.008, <emph>N</emph> = 23, <emph>p</emph> =.014, <emph>Phi (φ) =</emph>.511), indicating that the proportions of correct and incorrect assignments were significantly different between those in which the students' appraisals of definition was "ambivalent" and appraised as "accepted." In the 4 correct assignments, the students' appraisal of definition was "accepted" 1 (25%), which was smaller than the occurrences of "ambivalent" 3 (75%). In the 19 incorrect assignments, the students' appraisal of definition was accepted in 16 (60%) and "ambivalent" in 3 (40%).</p> <hd id="AN0185256566-24">Sequence three: Learning outcomes by the appraisal of definition</hd> <p>In sequence three, the instructor-prompted ChatGPT-produced text was provided at the beginning of instructional sequence. Results indicated that the proportion of correct assignments in which the students' appraisal of ChatGPT-produced text was either "accepted" or "ambivalent" was larger than "rejected" (Table 4).</p> <p>Table 4. Chi-square comparison tests for definition in sequence 3.</p> <p> <ephtml> <table><thead><tr><td>GPT info-quality as identified</td><td>Learning outcomes</td><td>Chi-square results</td></tr><tr><td>0<sup>a</sup></td><td>1<sup>b</sup></td><td><italic>χ<sup>2</sup></italic></td><td><italic>p</italic></td><td><italic>Phi (φ)</italic><xref ref-type="table-fn" rid="tfn10">*</xref></td></tr></thead><tbody valign="top"><tr><td /><td>(<italic>n</italic> = 4)</td><td>(<italic>n</italic> = 16)</td><td /><td /><td /></tr><tr><td>Rejected</td><td char=".">2 (50%)</td><td char=".">1 (6.2%)</td><td char=".">4.804</td><td char=".">.028</td><td char=".">.490</td></tr><tr><td>Ambivalent</td><td char=".">2 (50%)</td><td char=".">15 (93.8%)</td><td /><td /><td /></tr><tr><td /><td>(<italic>n</italic> = 2)</td><td>(<italic>n</italic> = 7)</td><td /><td /><td /></tr><tr><td>Rejected</td><td char=".">2 (100%)</td><td char=".">1 (14.2%)</td><td char=".">5.143</td><td char=".">.023</td><td char=".">.756</td></tr><tr><td>Accepted</td><td char=".">0 (0 %)</td><td char=".">6 (85.8%)</td><td /><td /><td /></tr><tr><td /><td>(<italic>n</italic> = 2)</td><td>(<italic>n</italic> = 21)</td><td /><td /><td /></tr><tr><td>Ambivalent</td><td char=".">2 (100%)</td><td char=".">15 (71.4%)</td><td char=".">0.773</td><td char=".">.379</td><td char=".">.183</td></tr><tr><td>Accepted</td><td char=".">0 (0 %)</td><td char=".">6 (28.6%)</td><td /><td /><td /></tr></tbody></table> </ephtml> </p> <ulist> <item>9 <emph>Notes:</emph></item> <item>10 All the <emph>Phi (φ)</emph> tests had the same significant level of <emph>p</emph> as in each of the <emph>χ<sups>2</sups></emph> tests.</item> <item>11 (<sups>1b</sups>) Assignments that the academic concepts were correctly formulated.</item> <item>12 (<sups>0a</sups>) Assignments that the academic concepts were not correctly formulated.</item> </ulist> <p>For example, the results of the first Chi-Square (<emph>χ<sups>2</sups></emph>) test were significant (<emph>χ<sups>2</sups></emph> = 4.804, <emph>N</emph> = 20, <emph>p</emph> =.028, <emph>Phi (φ) =</emph>.490), indicating that the proportions of correct assignments and incorrect assignments were significantly different between those in which the student's appraisal of definition was "ambivalent" and "rejected." In the 16 correct assignments, the students' appraisal of definition was "ambivalent" in 15 (93.8%), which was higher than the presence of rejections 1 (6.2%). In the four incorrect assignments, the appraisals were 2 (50%) as "ambivalent" and 2 (50%) as "rejected" on the definition.</p> <hd id="AN0185256566-25">Sequence three: Learning outcomes by the appraisal of application</hd> <p>In sequence three, the instructor-prompted ChatGPT-produced text was provided at the beginning of instructional task. Results indicated that the proportion of correct assignments in which the students' appraisal of ChatGPT-produced text was identified as either "accepted" or "ambivalent" was larger than those which were "rejected" (Table 5).</p> <p>Table 5. Chi-square comparison tests for application in sequence 3.</p> <p> <ephtml> <table><thead><tr><td>GPT info-quality as identified</td><td>Learning outcomes</td><td>Chi-square results</td></tr><tr><td>0<sup>a</sup></td><td>1<sup>b</sup></td><td><italic>χ<sup>2</sup></italic></td><td><italic>p</italic></td><td><italic>Phi (φ)</italic><xref ref-type="table-fn" rid="tfn14">*</xref></td></tr></thead><tbody valign="top"><tr><td /><td>(<italic>n</italic> = 6)</td><td>(<italic>n</italic> = 9)</td><td /><td /><td /></tr><tr><td>Rejected</td><td char=".">4 (66.7%)</td><td char=".">0 (0%)</td><td char=".">8.182</td><td char=".">.004</td><td char=".">.739</td></tr><tr><td>Ambivalent</td><td char=".">2 (33.3 %)</td><td char=".">9 (100%)</td><td /><td /><td /></tr><tr><td /><td>(<italic>n</italic> = 5)</td><td>(<italic>n</italic> = 11)</td><td /><td /><td /></tr><tr><td>Rejected</td><td char=".">4 (80%)</td><td char=".">2 (18.2%)</td><td char=".">5.605</td><td char=".">.018</td><td char=".">.592</td></tr><tr><td>Accepted</td><td char=".">1 (20 %)</td><td char=".">9 (81.8%)</td><td /><td /><td /></tr><tr><td /><td>(<italic>n</italic> = 1)</td><td>(<italic>n</italic> = 18)</td><td /><td /><td /></tr><tr><td>Ambivalent</td><td char=".">0 (0%)</td><td char=".">9 (50%)</td><td char=".">0.950</td><td char=".">.330</td><td char=".">.224</td></tr><tr><td>Accepted</td><td char=".">1 (100%)</td><td char=".">9 (50%)</td><td /><td /><td /></tr></tbody></table> </ephtml> </p> <ulist> <item>13 <emph>Notes:</emph></item> <item>14 All the <emph>Phi (φ)</emph> tests had the same significant level of <emph>p</emph> as in each of the <emph>χ<sups>2</sups></emph> tests.</item> <item>15 (<sups>1b</sups>) Assignments that the academic concepts were correctly formulated.</item> <item>16 (<sups>0a</sups>) Assignments that the academic concepts were not correctly formulated.</item> </ulist> <p>For example, the results of the second Chi-Square (<emph>χ<sups>2</sups></emph>) test were significant (<emph>χ<sups>2</sups></emph>= 5.605, <emph>N</emph> = 16, <emph>p</emph> =.018, Phi (φ) =.592), indicating that the proportions of correct assignments and incorrect assignments were significantly different between those in which the ChatGPT-produced text was appraised as "accepted" and as "rejected." In the 11 correct assignments, nine (81.8%) were appraisals of application were "accepted," which was higher than the two (18.2%) that were appraised as "rejected." The five incorrect assignments included 1 (20%) as "accepted" and four (80%) as "rejected" on the appraisal of application.</p> <hd id="AN0185256566-26">Summary of the findings</hd> <p></p> <ulist> <item> Student appraisal of <emph>definition</emph> in ChatGPT-produced text is associated with expected learning outcomes that the academic concepts were correctly formulated in all three instructional sequences:</item> <p></p> <item> In sequence one (instructor-prompted), the proportion of correct assignments in which the ChatGPT-produced definition was identified as "accepted" was larger than the proportion identified as "rejected" or "ambivalent."</item> <p></p> <item> In sequence two (student-prompted), the proportion of correct assignments in which the ChatGPT-produced definition was identified as "accepted" was smaller than the proportion identified as "rejected" or "ambivalent."</item> <p></p> <item> In sequence three (instructor-prompted), the proportion of correct assignments in which the ChatGPT-produced definition was identified as either "accepted" or "ambivalent" was larger than the proportion identified as "rejected".</item> <p></p> <item> Student appraisals of <emph>application</emph> in ChatGPT-produced text is associated with expected learning outcomes when the academic concepts were correctly formulated in learning sequence three:</item> <p></p> <item> In sequence three (instructor-prompted), the proportion of correct assignments in which the ChatGPT-produced application identified as either "accepted" or "ambivalent" was larger than the proportion identified as "rejected".</item> </ulist> <hd id="AN0185256566-27">Summary of effect size</hd> <p>In Chi-square test, effect size measurements <emph>Phi φ</emph> (for a two-by-two design) and <emph>Cramer's V</emph> when the contingency table is larger than a two-by-two design and are used to quantify the strength of association between two categorical variables. The effect size values of (.1), (.3), and (.5) are considered small, medium and large, respectively (Cohen, [<reflink idref="bib7" id="ref119">7</reflink>]). In this study, the <emph>Cramer's V</emph> values ranged from (.620) to (.806) for tests that were significant in research question one, and the <emph>Phi (φ)</emph> values ranged from (.490) to (.756) for Chi-square tests that were significant in research question two, indicating medium to large effect sizes. They were also consistent with the expected effect sizes set in the priori power analysis, ranged from (.50) to (.65), determining the range of the sample size for this study.</p> <p>The effect sizes in relevant studies were reviewed. Sehlaoui and Shinge ([<reflink idref="bib34" id="ref120">34</reflink>]) conducted chi-square tests to examine the knowledge in applied linguistics for English language teachers and the effect size <emph>Phi (φ)</emph> values ranged from.243 to.301 (<emph>N</emph> = 201). Payaprom and Payaprom ([<reflink idref="bib31" id="ref121">31</reflink>]) examined language learners' learning style by majors, and the <emph>Cramer's V</emph>was.15 (<emph>N</emph> = 372). Another study on teacher education investigating the extent to which learning style associated with gender and type of education (Can, [<reflink idref="bib4" id="ref122">4</reflink>]) revealed the calculated<emph>Cramer's V</emph> values of.20 and.21 (Ben et al., [<reflink idref="bib3" id="ref123">3</reflink>]). Compared with relevant studies in the literature, effect sizes in this study were relatively larger than those from the literature.</p> <hd id="AN0185256566-28">Discussion and conclusion</hd> <p>Set within Vygotsky's ([<reflink idref="bib39" id="ref124">39</reflink>]) sociocultural framework, this study builds upon on previous research confirming the potential of ChatGPT-produced text to enhance preservice teachers' understanding of pedagogical content toward the use of academic concepts (Author & Author, [<reflink idref="bib2" id="ref125">2</reflink>], under review). The content analysis suggests that the students recognized the potential of ChatGPT to produce accurate and even useful definitions but doubted its ability to produce instruction (application) responsive to the cultural and linguistic needs of multilingual students. In their appraisals of ChatGPT-produced texts, they emphasized the importance of the human element in instruction and questioned whether or not ChatGPT was capable of producing instruction without a teacher's guidance. How the findings reflect the tension between definition and their thoughts about the importance of the human element in instruction (application) is explored below.</p> <p>Before a discussion of the findings, it is necessary to address the limitations of this exploratory study. First, the study employed a nonrandomized within-subject design, using two sections of an existing class with a relatively small sample size (<emph>N</emph> = 30). The results and findings are not generalizable to larger populations. Second, three instructional sequences and three learning tasks were examined in this study. The ChatGPT-produced texts including definition, application, and technical concerns were based on particular prompts related only to these instructional sequences. The interpretation of the results and findings was limited to the context of this exploratory study. Drawing on firsthand experiences of preservice teachers, this exploratory study suggests practical means of ChatGPT-integrated learning for further exploration. Further studies would benefit from empirical design with a larger random sample, and with a variety of means of ChatGPT integrated methods, procedures, sequences, learning conditions and learning tasks.</p> <hd id="AN0185256566-29">Overall appraisal of ChatGPT-produced text</hd> <p>With respect to research question one, findings suggest that the overall appraisal of definition was associated with students' correct use of the academic concept across all three sequences. In the instructor-prompted instructional sequences one and three, among preservice teachers who achieved an academic concept in their assignment, findings reveal a higher proportion of students either accepted or were ambivalent toward the use of ChatGPT-produced definition (Tables 2 and 4). Ambivalent appraisals, again across all sequences, typically balanced praise for its rhetorical and organizational skills with questions about its inherent inability to factor emotion, culture, and human experience into its explanations. An example from sequence one captures the sentiment.</p> <p>With regards to [its effectiveness], I think ChatGPT was able to create a coherent essay that answered the prompt. However, I also believe that there was a lack of serious analysis, reflecting a lack of ability to understand the relationship between language and culture, since this is an experience that is extremely human in nature. Although it is able to think in language in the same way we do, it cannot apply that language in the same way, because of the fact that it does not have a human experience.</p> <p>The student's praise for 'coherent' writing in ChatGPT-produced text is supported by Katz et al. ([<reflink idref="bib19" id="ref126">19</reflink>]) and Gilson et al. ([<reflink idref="bib12" id="ref127">12</reflink>]), but her ambivalence also suggests a growing awareness that education is a human enterprise. She argues that "serious analysis" of the linguistic and cultural issues that fuel an understanding of the instruction of multilingual learners excludes ChatGPT-produced text. Throughout the remaining assignments, students would use ChatGPT-produced text as the straw dummy in the debate over whether or not human experience is prerequisite to understanding the instruction of multilingual students.</p> <hd id="AN0185256566-30">Further exploration of appraisals in ChatGPT-produced text</hd> <p>With respect to research question two, findings from sequence three suggest at least two important points. First, careful preparation and strategic use of ChatGPT-produced content can widen students' appraisals of ChatGPT-produced content of both definition and application. Specifically, placing ChatGPT-produced text that is teacher-prompted at the beginning of the instructional task can inspire appraisal of both definition and application, which, potentially, could lead to a broad understanding of the academic concept. The association between the correct use of the academic concept and appraisal of both definition (<emph>χ<sups>2</sups></emph>= 10.887, <emph>p</emph> =.012) and application (<emph>χ<sups>2</sups></emph> = 17.532, <emph>p</emph> <.001) was significant only in sequence three.</p> <p>Second, findings from sequence three instigated research question two, a deeper examination of the students' appraisals from the three sequences. Findings revealed that among students who reached an academic concept in their writing, a significantly higher proportion either accepted use of ChatGPT, (81.8%), compared with those who rejected (18.2%), or were ambivalent about its use. In the instructor-prompted instructional sequences one and three, among students who achieved an academic concept in their assignment, findings revealed a higher proportion of students who had either accepted or were ambivalent to use ChatGPT-produced definition (Tables 2 and 4). However, in sequence two, students who did not reach an academic concept either accepted (94.1%) or were ambivalent (75%) to use the ChatGPT-produced definition. In contrast, those who achieved the academic concepts identified a lower proportion for accepted (25%) and ambivalent (50%).</p> <p>One explanation for the wider variation found in sequence two might be that it is the only student-prompted sequence. Students may be entering prompts of varying quality into ChatGPT, resulting in ChatGPT-produced texts of varying quality that may or may not be relevant to the correct academic concept. This is consistent with research on the influence of prompts on the quality of ChatGPT-produced texts. Gilson et al. ([<reflink idref="bib12" id="ref128">12</reflink>]), for instance, noted that ChatGPT is not able to interpret tables or figures and provides higher quality responses when given sample questions. The data did not show evidence that the preservice teachers were aware of this. While some discovered that it took multiple prompts to receive a satisfactory response from ChatGPT, there was no evidence of them questioning the responses because of ChatGPT's inability to interpret tables of figures.</p> <p>In conclusion, ChatGPT-produced texts provide a valuable resource in LTE, but its use requires the guiding hand of the teacher educator. As such, learning with ChatGPT-produced text begins with teacher-prompted instruction and attention to its placement within the instructional sequence. As the more knowledgeable other, the teacher educator is best equipped to assess the extent to which ChatGPT articulates an understanding of the selected academic concept and employ the use of ChatGPT-produced text to open discussions which expand students' thinking about the human element in instruction.</p> <hd id="AN0185256566-31">Implications for practice</hd> <p>The findings suggest a place for the use of ChatGPT within a sociocultural framework and the need to consider how ChatGPT can be used in a socially and culturally mediated environment. Vygotsky's ([<reflink idref="bib39" id="ref129">39</reflink>]) theory demonstrates that learning is a socially-situated experience in which cultural tools, which include language and technology, enhance cognitive development. For the teacher educator, the introduction of ChatGPT presents new opportunities to create structured mediated spaces which challenge preservice teachers to advance their understanding of pedagogical content toward an academic concept. Throughout the process, preservice teachers come to see ChatGPT as a valuable tool which mediates their learning but also requires them to actively critique its content. Exploring the tension between these two roles provides a powerful forum for preservice teachers to examine their own emerging understanding of pedagogy, ultimately preparing them to integrate AI-driven technology into their own instruction.</p> <p>Teacher educators can begin with prompting preservice teachers to examine their own their current perceptions of ChatGPT. Attitudes about ChatGPT are predictive of the frequency and ways that preservice teachers use ChatGPT in the classroom (Javier & Moorhouse, [<reflink idref="bib16" id="ref130">16</reflink>]; Moorhouse, [<reflink idref="bib28" id="ref131">28</reflink>]; Moorhouse & Kohnke, [<reflink idref="bib29" id="ref132">29</reflink>]; Yang & Appleget, [<reflink idref="bib42" id="ref133">42</reflink>]). Because many may not have extensive experience with ChatGPT (Moorhouse, [<reflink idref="bib28" id="ref134">28</reflink>]), opportunities to read about its potential to create lessons and classroom exercises (e.g., Farrokhnia et al., [<reflink idref="bib11" id="ref135">11</reflink>]; Skrabut, [<reflink idref="bib36" id="ref136">36</reflink>]) can be valuable. Teacher educators can prompt preservice teachers to reflect on its potential in the classroom in class discussions. The experience provides what Johnson and Golombek ([<reflink idref="bib18" id="ref137">18</reflink>]) refer to as a play/stepping into teaching, as it provides a space for preservice teachers to try out new ideas in a safe space under the guidance of a more knowledgeable other.</p> <p>Next, teacher educators can problematize the content and use of ChatGPT by asking them to explore the interplay between technology and human experience. This might be done in a number of ways. Examples from this study included critiquing ChatGPT-created lessons and essays, asking preservice teachers to compare ChatGPT-produced essays on pedagogical topics with their own writing and responding to ChatGPT-produced posts in discussion boards. Next, teacher educators can press students to work independently with ChatGPT to create lessons and pedagogical content while still providing guidance as the knowledgeable other. Finally, more research is needed on the ways in which ChatGPT can provide a forum for how teaching is a human-centered occupation.</p> <p>These findings are a call for research on the nature and potential use of ChatGPT-produced texts in the teacher education classroom. This study spoke to the importance of teacher-guided experiences when introducing ChatGPT-produced texts, but the preservice teachers in this study were enrolled in their first and only class on teaching multilingual learners. This opens questions for future research on preservice teachers who are more knowledgeable of pedagogical content and LTE. Do they require the same level of scaffolding when using ChatGPT-produced texts? If so, how should that scaffolding proceed? Finally, this study also raised questions about the extent to which ChatGPT is able produce text which is reflective of multilingual students' culture and learning experiences. Students identified this as the human element, and it is important divide that has to be better understood as teacher educators move forward within integrating AI-based texts and materials into their classrooms. Researchers might continue to explore the ways that ChatGPT-produced lessons can inspire discussions and learning on the importance of the human element in instruction but there are many more avenues which could both incorporate preservice teachers' experiences in K-12 classrooms and teacher education courses.</p> <p>The introduction of ChatGPT in 2022 and subsequent research in teacher education regarding its classroom possibilities represent a clarion call for teacher educators to engage in a dialogue on its use. Framed within a sociocultural framework, the use of ChatGPT-produced text in LTE should be leveraged to advance preservice students toward a deeper understanding of pedagogical content. More broadly, this study encourages a curriculum and instructional practices which encourage preservice teachers to confront how language, culture and human experience shape the instruction of multilingual students. Achieving this will push teacher educators to integrate ChatGPT in ways that go beyond the polemics of "ChatGPT versus human instruction," fostering a deeper discussion about the interconnection of language, culture, and human experience.</p> <hd id="AN0185256566-32">Disclosure statement</hd> <p>No potential conflict of interest was reported by the author(s).</p> <ref id="AN0185256566-33"> <title> References </title> <blist> <bibl id="bib1" idref="ref63" type="bt">1</bibl> <bibtext> Author (2024).</bibtext> </blist> <blist> <bibl id="bib2" idref="ref21" type="bt">2</bibl> <bibtext> Author, A., & Author, B. (2024).</bibtext> </blist> <blist> <bibl id="bib3" idref="ref111" type="bt">3</bibl> <bibtext> Ben, Shachar, M. S., Patil, I., Thériault, R., Wiernik, B. M., & Lüdecke, D. (2023). Phi, Fei, Fo, Fum: Effect sizes for categorical data that use the chi‑squared statistic. Mathematics, 11 (9), 1982. 2023, https://doi.org/10.3390/math11091982</bibtext> </blist> <blist> <bibl id="bib4" idref="ref122" type="bt">4</bibl> <bibtext> Can, S. (2009). The effects of science student teachers' academic achievements, their grade levels, gender and type of education they are exposed to on their 4mat learning styles. Procedia - Social and Behavioral Sciences, 1 (1), 1853 – 1857. https://doi.org/10.1016/j.sbspro.2009.01.327</bibtext> </blist> <blist> <bibl id="bib5" idref="ref38" type="bt">5</bibl> <bibtext> Ciechanowski, L., Przegalinska, A., Magnuski, M., & Gloor, P. (2019). In the shades of the uncanny valley: An experimental study of human–chatbot interaction. Future Generation Computer Systems, 92, 539 – 548. https://doi.org/10.1016/j.future.2018.01.055</bibtext> </blist> <blist> <bibl id="bib6" idref="ref117" type="bt">6</bibl> <bibtext> Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20 (1), 37 – 46. https://doi.org/10.1177/001316446002000104</bibtext> </blist> <blist> <bibl id="bib7" idref="ref119" type="bt">7</bibl> <bibtext> Cohen, J. (2013). Statistical power analysis for the behavioral sciences Routledge. https://doi.org/10.4324/9780203771587</bibtext> </blist> <blist> <bibl id="bib8" idref="ref40" type="bt">8</bibl> <bibtext> Cooper, G. (2023). Examining science education in CHATGPT: An exploratory study of Generative Artificial Intelligence. Journal of Science Education and Technology, 32 (3), 444 – 452. https://doi.org/10.1007/s10956-023-10039-y</bibtext> </blist> <blist> <bibl id="bib9" idref="ref90" type="bt">9</bibl> <bibtext> de Souza, M. (2017). Understanding Mexican immigrant students in American schools: A case study of two preparatorias in México. Cogent Education, 4 (1), 1387963. https://doi.org/10.1080/2331186X.2017.1387963</bibtext> </blist> <blist> <bibtext> Elkins, K., & Chun, J. (2020). Can GPT-3 pass a writer's Turing test? Journal of Cultural Analytics, 5 (2). https://doi.org/10.22148/001c.17212</bibtext> </blist> <blist> <bibtext> Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2024). A SWOT analysis of CHATGPT: Implications for educational practice and Research. Innovations in Education and Teaching International, 61 (3), 460 – 474. https://doi.org/10.1080/14703297.2023.2195846</bibtext> </blist> <blist> <bibtext> Gilson, A., Safranek, C., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2022). How does ChatGPT perform on the medical licensing exams? The implications of large language models for medical education and knowledge assessment. https://doi.org/10.1101/2022.12.23.22283901</bibtext> </blist> <blist> <bibtext> Goldsmith, J. S., & Kurpius, S. E. (2018). Fostering the academic success of their children: Voices of Mexican immigrant parents. The Journal of Educational Research, 111 (5), 564 – 573. https://doi.org/10.1080/00220671.2017.1323717</bibtext> </blist> <blist> <bibtext> Hanselman, K, Author. (2021).</bibtext> </blist> <blist> <bibtext> Huh, S. (2023). Are ChatGPT's knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination? A descriptive study. Journal of Educational Evaluation for Health Professions, 20, 1. https://doi.org/10.3352/jeehp.2023.20.1</bibtext> </blist> <blist> <bibtext> Javier, D. R., & Moorhouse, B. L. (2024). Developing secondary school English language learners' productive and critical use of ChatGPT. TESOL Journal, 15 (2). https://doi.org/10.1002/tesj.755</bibtext> </blist> <blist> <bibtext> Jeon, J., & Lee, S. (2023). Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT. Education and Information Technologies, 28 (12), 15873 – 15892. https://doi.org/10.1007/s10639-023-11834-1</bibtext> </blist> <blist> <bibtext> Johnson, K. E., & Golombek, P. R. (2020). Informing and transforming language teacher education pedagogy. Language Teaching Research, 24 (1), 116 – 127. https://doi.org/10.1177/1362168818777539</bibtext> </blist> <blist> <bibtext> Katz, D. M., Bommarito, M. J., Gao, S., & Arredondo, P. (2023). GPT-4 passes the bar exam. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4389233</bibtext> </blist> <blist> <bibtext> Khalaf, M. A. (2024). Does attitude towards plagiarism predict Aigiarism using ChatGPT? AI and Ethics. https://doi.org/10.1007/s43681-024-00426-5</bibtext> </blist> <blist> <bibtext> Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. RELC Journal, 54 (2), 537 – 550. https://doi.org/10.1177/00336882231162868</bibtext> </blist> <blist> <bibtext> Kostka, I., & Toncelli, R, Northeastern University (2023). Exploring applications of ChatGPT to English language teaching: Opportunities, challenges, and recommendations. Teaching English as a Second or Foreign Language-TESL-EJ, 27 (3). https://doi.org/10.55593/ej.27107int</bibtext> </blist> <blist> <bibtext> Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33 (1), 159 – 174. https://doi.org/10.1080/10543406.2016.1273941</bibtext> </blist> <blist> <bibtext> Lantolf, J. P., Xi, J., & Minakova, V. (2021). Sociocultural theory and concept-based language instruction. Language Teaching, 54 (3), 327 – 342. https://doi.org/10.1017/S0261444820000348</bibtext> </blist> <blist> <bibtext> Li, T., Ji, Y., & Zhan, Z. (2024). Expert or machine? comparing the effect of pairing student teacher with in-service teacher and CHATGPT on their critical thinking, learning performance, and cognitive load in an integrated-stem course. Asia Pacific Journal of Education, 44 (1), 45 – 60. https://doi.org/10.1080/02188791.2024.2305163</bibtext> </blist> <blist> <bibtext> Liu, L. (2023). Analyzing the text content produced by ChatGPT: Prompts, feature-components in responses, and a predictive model. Journal of Educational Technology Development and Exchange, 16 (1), 49 – 70. https://doi.org/10.18785/jetde.1601.03</bibtext> </blist> <blist> <bibtext> Moll, L. C. (2019). Elaborating funds of knowledge: Community-oriented practices in international contexts. Literacy Research: Theory, Method, and Practice, 68 (1), 130 – 138. https://doi.org/10.1177/2381336919870805</bibtext> </blist> <blist> <bibtext> Moorhouse, B. L. (2024). Beginning and first-year language teachers' readiness for the generative AI age. Computers and Education: Artificial Intelligence, 6, 100201. https://doi.org/10.1016/j.caeai.2024.100201</bibtext> </blist> <blist> <bibtext> Moorhouse, B. L., & Kohnke, L. (2024). The effects of generative AI on initial language teacher education: The perceptions of teacher educators. System, 122, 103290. https://doi.org/10.1016/j.system.2024.103290</bibtext> </blist> <blist> <bibtext> Nicolaidou, I., Pissas, P., & Boglou, D. (2023). Comparing immersive Virtual Reality to mobile applications in foreign language learning in higher education: A quasi-experiment. Interactive Learning Environments, 31 (4), 2001 – 2015. https://doi.org/10.1080/10494820.2020.1870504</bibtext> </blist> <blist> <bibtext> Payaprom, S., & Payaprom, Y. (2020). Identifying learning styles of language learners: A useful step in moving towards the learner-centred approach. Journal of Language and Linguistic Studies, 16 (1), 59 – 72. https://doi.org/10.17263/jlls.712646</bibtext> </blist> <blist> <bibtext> Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Assessing social presence in asynchronous, text-based computer conferencing. Journal of Distance Education, 14 (3), 51 – 70. Retrieved October10, 2024 from https://<ulink href="http://www.ijede.ca/index.php/jde/article/view/153">www.ijede.ca/index.php/jde/article/view/153</ulink></bibtext> </blist> <blist> <bibtext> Rusandi, M Arli, Saripah, Ipah, Khairun, Deasy Yunika, Ahman, Mutmainnah, (2023). No worries with CHATGPT: Building bridges between artificial intelligence and education with critical thinking soft skills. Journal of Public Health (Oxford, England), 45 (3, e602 – e603). https://doi.org/10.1093/pubmed/fdad049</bibtext> </blist> <blist> <bibtext> Sehlaoui, A. S., & Shinge, M. (2013). The pursuit of quality over quantity in TESOL teacher education: Coursework Versus test only. TESOL Journal, 4 (1), 106 – 128. https://doi.org/10.1002/tesj.53</bibtext> </blist> <blist> <bibtext> Shabani, K. (2016). Applications of Vygotsky's sociocultural approach for teachers' professional development. Cogent Education, 3 (1), 1252177. https://doi.org/10.1080/2331186X.2016.1252177</bibtext> </blist> <blist> <bibtext> Skrabut, S. (2023). 80 ways to use ChatGPT in the classroom: Using AI to enhance teaching and learning. Sandman Books.</bibtext> </blist> <blist> <bibtext> Solorza, C., Aponte, g., Leverenz, T., Becker, T., & Frias, B. (2023). (rep.). Translanguaging in dual language bilingual education: A blueprint for planning units of study (pp. 8 – 88). CUNY-NYSIEB, The Graduate Center, The City University of New York.</bibtext> </blist> <blist> <bibtext> van den Berg, G., & Du Plessis, E. (2023). CHATGPT and Generative AI: Possibilities for its contribution to lesson planning, critical thinking and openness in teacher education. Education Sciences, 13 (10), 998. https://doi.org/10.3390/educsci13100998</bibtext> </blist> <blist> <bibtext> Vygotsky, L. S. (1986). Thought and language. Translated by A. Kozulin MIT Press.</bibtext> </blist> <blist> <bibtext> Weber, R. P. (1990). Basic content analysis Sage Publications.</bibtext> </blist> <blist> <bibtext> Williamson, M. (2024). Sample size calculation with GPower. Retrieved June 20, 2024 from: https://med.und.edu/research/daccota/_files/pdfs/berdc_resource_pdfs/sample_size_gpower_module.pdf</bibtext> </blist> <blist> <bibtext> Yang, S., & Appleget, C. (2024). An exploration of preservice teachers' perceptions of generative AI: Applying the technological acceptance model. Journal of Digital Learning in Teacher Education, 40 (3), 159 – 172. https://doi.org/10.1080/21532974.2024.2367573</bibtext> </blist> <blist> <bibtext> Yeadon, W., Inyang, O.-O., Mizouri, A., Peach, A., & Testrow, C. P. (2023). The death of the short-form physics essay in the Coming AI Revolution. Physics Education, 58 (3), 035027. https://doi.org/10.1088/1361-6552/acc5cf</bibtext> </blist> <blist> <bibtext> Zhai, X. (2022). CHATGPT user experience: Implications for education. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4312418</bibtext> </blist> <blist> <bibtext> Zwiers, J. (2014). Building academic language: Meeting common core standards across disciplines, grades 5–12. Jossey-Bass.</bibtext> </blist> </ref> <aug> <p>By Rod E. Case and Leping Liu</p> <p>Reported by Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib16" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib28" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib29" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib42" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib11" firstref="ref5"></nolink> <nolink nlid="nl6" bibid="bib25" firstref="ref6"></nolink> <nolink nlid="nl7" bibid="bib38" firstref="ref7"></nolink> <nolink nlid="nl8" bibid="bib21" firstref="ref9"></nolink> <nolink nlid="nl9" bibid="bib22" firstref="ref10"></nolink> <nolink nlid="nl10" bibid="bib12" firstref="ref11"></nolink> <nolink nlid="nl11" bibid="bib15" firstref="ref12"></nolink> <nolink nlid="nl12" bibid="bib43" firstref="ref13"></nolink> <nolink nlid="nl13" bibid="bib19" firstref="ref14"></nolink> <nolink nlid="nl14" bibid="bib10" firstref="ref15"></nolink> <nolink nlid="nl15" bibid="bib20" firstref="ref20"></nolink> <nolink nlid="nl16" bibid="bib40" firstref="ref25"></nolink> <nolink nlid="nl17" bibid="bib33" firstref="ref37"></nolink> <nolink nlid="nl18" bibid="bib36" firstref="ref42"></nolink> <nolink nlid="nl19" bibid="bib17" firstref="ref44"></nolink> <nolink nlid="nl20" bibid="bib44" firstref="ref61"></nolink> <nolink nlid="nl21" bibid="bib35" firstref="ref67"></nolink> <nolink nlid="nl22" bibid="bib39" firstref="ref68"></nolink> <nolink nlid="nl23" bibid="bib18" firstref="ref69"></nolink> <nolink nlid="nl24" bibid="bib24" firstref="ref76"></nolink> <nolink nlid="nl25" bibid="bib30" firstref="ref87"></nolink> <nolink nlid="nl26" bibid="bib41" firstref="ref88"></nolink> <nolink nlid="nl27" bibid="bib26" firstref="ref89"></nolink> <nolink nlid="nl28" bibid="bib27" firstref="ref91"></nolink> <nolink nlid="nl29" bibid="bib13" firstref="ref93"></nolink> <nolink nlid="nl30" bibid="bib45" firstref="ref94"></nolink> <nolink nlid="nl31" bibid="bib37" firstref="ref95"></nolink> <nolink nlid="nl32" bibid="bib32" firstref="ref113"></nolink> <nolink nlid="nl33" bibid="bib14" firstref="ref115"></nolink> <nolink nlid="nl34" bibid="bib23" firstref="ref116"></nolink> <nolink nlid="nl35" bibid="bib34" firstref="ref120"></nolink> <nolink nlid="nl36" bibid="bib31" firstref="ref121"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1493362
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: ChatGPT-Produced Content as a Resource in the Language Education Classroom: A Guiding Hand
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Rod+E%2E+Case%22">Rod E. Case</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1175-648X">0000-0003-1175-648X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Leping+Liu%22">Leping Liu</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5859-8189">0000-0001-5859-8189</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Computers+in+the+Schools%22"><i>Computers in the Schools</i></searchLink>. 2025 42(2):187-211.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 25
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Preservice+Teachers%22">Preservice Teachers</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Evaluation%22">Writing Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Teachers%22">Language Teachers</searchLink><br /><searchLink fieldCode="DE" term="%22State+Universities%22">State Universities</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+%28Composition%29%22">Writing (Composition)</searchLink><br /><searchLink fieldCode="DE" term="%22Preservice+Teacher+Education%22">Preservice Teacher Education</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Education+Programs%22">Teacher Education Programs</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Attitudes%22">Student Attitudes</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/07380569.2024.2442700
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0738-0569<br />1528-7033
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: While there is much in teacher education which examines the perceptions of preservice teacher to the recent introduction of ChatGPT. This study examines preservice teachers' appraisals of ChatGPT-produced text. Using a quasi-experimental within-­subject design with 30 participants, Chi-Square tests examined appraisals under three conditions. Results indicated that appraisal of ChatGPT-produced text was higher when prepared by the instructor among students who demonstrated an understanding of the pedagogical content versus those who did not. Appraisals were also sensitive to where ChatGPT-produced text was placed in the assignment, favoring use of ChatGPT at beginning of the assignment versus in the middle. A discussion of implications for using ChatGPT-produced text is included.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1493362
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1493362
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/07380569.2024.2442700
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 25
        StartPage: 187
    Subjects:
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Preservice Teachers
        Type: general
      – SubjectFull: Writing Evaluation
        Type: general
      – SubjectFull: Technology Uses in Education
        Type: general
      – SubjectFull: Language Teachers
        Type: general
      – SubjectFull: State Universities
        Type: general
      – SubjectFull: Writing (Composition)
        Type: general
      – SubjectFull: Preservice Teacher Education
        Type: general
      – SubjectFull: Teacher Education Programs
        Type: general
      – SubjectFull: Student Attitudes
        Type: general
    Titles:
      – TitleFull: ChatGPT-Produced Content as a Resource in the Language Education Classroom: A Guiding Hand
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Rod E. Case
      – PersonEntity:
          Name:
            NameFull: Leping Liu
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0738-0569
            – Type: issn-electronic
              Value: 1528-7033
          Numbering:
            – Type: volume
              Value: 42
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Computers in the Schools
              Type: main
ResultId 1