Natural Language Processing as a Scalable Method for Evaluating Educational Text Personalization by LLMs
Saved in:
| Title: | Natural Language Processing as a Scalable Method for Evaluating Educational Text Personalization by LLMs |
|---|---|
| Language: | English |
| Authors: | Linh Huynh (ORCID |
| Source: | Grantee Submission. 2025 15. |
| Peer Reviewed: | Y |
| Page Count: | 27 |
| Publication Date: | 2025 |
| Sponsoring Agency: | National Center for Education Research (NCER) (ED/IES) |
| Contract Number: | R305T240035 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Artificial Intelligence, Natural Language Processing, Textbook Evaluation, Individualized Instruction, Reading Skills, Knowledge Level, Language Variation, Reading Materials, Science Materials, History, Student Needs, Evaluation Methods |
| DOI: | 10.3390/app152212128 |
| Abstract: | Four versions of science and history texts were tailored to diverse hypothetical reader profiles (high and low reading skills and domain knowledge), generated by four Large Language Models (i.e., Claude, Llama, ChatGPT, and Gemini). The Natural Language Processing (NLP) technique was applied to examine variations in Large Language Model (LLM) text personalization capabilities. NLP was leveraged to extract and quantify linguistic features of these texts, capturing linguistic variations as a function of LLMs, text genres, and reader profiles. An approach leveraging NLP-based analyses provides an automated and scalable solution for evaluating alignment between LLM-generated personalized texts and readers' needs. Findings indicate that NLP offers a valid and generalizable means of tracking linguistic variation in personalized educational texts, supporting its use as an evaluation framework for text personalization. |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2025 |
| Accession Number: | ED677279 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Four versions of science and history texts were tailored to diverse hypothetical reader profiles (high and low reading skills and domain knowledge), generated by four Large Language Models (i.e., Claude, Llama, ChatGPT, and Gemini). The Natural Language Processing (NLP) technique was applied to examine variations in Large Language Model (LLM) text personalization capabilities. NLP was leveraged to extract and quantify linguistic features of these texts, capturing linguistic variations as a function of LLMs, text genres, and reader profiles. An approach leveraging NLP-based analyses provides an automated and scalable solution for evaluating alignment between LLM-generated personalized texts and readers' needs. Findings indicate that NLP offers a valid and generalizable means of tracking linguistic variation in personalized educational texts, supporting its use as an evaluation framework for text personalization. |
|---|---|
| DOI: | 10.3390/app152212128 |
