Unlocking the potential of Socratic tutoring powered by large language model: A pilot study.
Saved in:
| Title: | Unlocking the potential of Socratic tutoring powered by large language model: A pilot study. |
|---|---|
| Authors: | Lang, Yueru1,2,3 langyueru_michelle@163.com, Hu, Xiangen4 xiangen.hu@polyu.edu.hk, Gong, Shaoying1,2,3 gongsy@ccnu.edu.cn, Wei, Huiling1,2,3 18154018518@163.com |
| Source: | Educational Technology & Society. Apr2026, Vol. 29 Issue 2, p69-83. 15p. |
| Subject Terms: | *Intelligent tutoring systems, *Tutors & tutoring, *Academic achievement, Sentiment analysis, User experience, Language models, Pilot projects, Generative pre-trained transformers |
| Abstract: | This pilot study introduced a conversational intelligent tutoring system named Socratic Playground for Learning (SPL) as a promising solution for simulating Socratic tutoring powered by GPT-4. It preliminarily investigated the effectiveness of Large Language Model-powered Socratic Tutoring (LLM-ST) in learning gains and user experience. Results showed that: SPL could provide learners with high-quality content. Participants achieved an average learning gain of 18.43%. Additionally, the user experience was generally positive during learning: participants experienced higher levels of positive emotions and lower levels of negative emotions, with a notable reduction in negative emotions following the learning phase. Log analysis revealed that learners rarely expressed positive and negative emotions during interactions with SPL, and confusion was the primary emotion when expressed. Semi-structured interviews with participants indicated that SPL was effective in facilitating learning (e.g., supporting knowledge mastery and inspiring thinking), while also gathering valuable feedback for further improvements (e.g., simplifying learning phases and providing detailed guidance). These findings suggest that LLM-ST has the potential to facilitate knowledge acquisition and provide a positive user experience. [ABSTRACT FROM AUTHOR] |
| Copyright of Educational Technology & Society is the property of International Forum of Educational Technology & Society (IFETS) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 192828622 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Unlocking the potential of Socratic tutoring powered by large language model: A pilot study. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Lang%2C+Yueru%22">Lang, Yueru</searchLink><relatesTo>1,2,3</relatesTo><i> langyueru_michelle@163.com</i><br /><searchLink fieldCode="AR" term="%22Hu%2C+Xiangen%22">Hu, Xiangen</searchLink><relatesTo>4</relatesTo><i> xiangen.hu@polyu.edu.hk</i><br /><searchLink fieldCode="AR" term="%22Gong%2C+Shaoying%22">Gong, Shaoying</searchLink><relatesTo>1,2,3</relatesTo><i> gongsy@ccnu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Wei%2C+Huiling%22">Wei, Huiling</searchLink><relatesTo>1,2,3</relatesTo><i> 18154018518@163.com</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Educational+Technology+%26+Society%22">Educational Technology & Society</searchLink>. Apr2026, Vol. 29 Issue 2, p69-83. 15p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Intelligent+tutoring+systems%22">Intelligent tutoring systems</searchLink><br />*<searchLink fieldCode="DE" term="%22Tutors+%26+tutoring%22">Tutors & tutoring</searchLink><br />*<searchLink fieldCode="DE" term="%22Academic+achievement%22">Academic achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Sentiment+analysis%22">Sentiment analysis</searchLink><br /><searchLink fieldCode="DE" term="%22User+experience%22">User experience</searchLink><br /><searchLink fieldCode="DE" term="%22Language+models%22">Language models</searchLink><br /><searchLink fieldCode="DE" term="%22Pilot+projects%22">Pilot projects</searchLink><br /><searchLink fieldCode="DE" term="%22Generative+pre-trained+transformers%22">Generative pre-trained transformers</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: This pilot study introduced a conversational intelligent tutoring system named Socratic Playground for Learning (SPL) as a promising solution for simulating Socratic tutoring powered by GPT-4. It preliminarily investigated the effectiveness of Large Language Model-powered Socratic Tutoring (LLM-ST) in learning gains and user experience. Results showed that: SPL could provide learners with high-quality content. Participants achieved an average learning gain of 18.43%. Additionally, the user experience was generally positive during learning: participants experienced higher levels of positive emotions and lower levels of negative emotions, with a notable reduction in negative emotions following the learning phase. Log analysis revealed that learners rarely expressed positive and negative emotions during interactions with SPL, and confusion was the primary emotion when expressed. Semi-structured interviews with participants indicated that SPL was effective in facilitating learning (e.g., supporting knowledge mastery and inspiring thinking), while also gathering valuable feedback for further improvements (e.g., simplifying learning phases and providing detailed guidance). These findings suggest that LLM-ST has the potential to facilitate knowledge acquisition and provide a positive user experience. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Educational Technology & Society is the property of International Forum of Educational Technology & Society (IFETS) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=192828622 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.30191/ETS.202604_29(2).RP04 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 15 StartPage: 69 Subjects: – SubjectFull: Intelligent tutoring systems Type: general – SubjectFull: Tutors & tutoring Type: general – SubjectFull: Academic achievement Type: general – SubjectFull: Sentiment analysis Type: general – SubjectFull: User experience Type: general – SubjectFull: Language models Type: general – SubjectFull: Pilot projects Type: general – SubjectFull: Generative pre-trained transformers Type: general Titles: – TitleFull: Unlocking the potential of Socratic tutoring powered by large language model: A pilot study. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Lang, Yueru – PersonEntity: Name: NameFull: Hu, Xiangen – PersonEntity: Name: NameFull: Gong, Shaoying – PersonEntity: Name: NameFull: Wei, Huiling IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 04 Text: Apr2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 11763647 Numbering: – Type: volume Value: 29 – Type: issue Value: 2 Titles: – TitleFull: Educational Technology & Society Type: main |
| ResultId | 1 |
