View in EDS

GenAI-Powered Text Personalization: Natural Language Processing Validation of Adaptation Capabilities

Saved in:
Bibliographic Details
Title: GenAI-Powered Text Personalization: Natural Language Processing Validation of Adaptation Capabilities
Language: English
Authors: Linh Huynh, Danielle S. McNamara (ORCID 0000-0001-5869-1420)
Source: Grantee Submission. 2025.
Peer Reviewed: Y
Page Count: 22
Publication Date: 2025
Sponsoring Agency: National Center for Education Research (NCER) (ED/IES)
Contract Number: R305T240035
Document Type: Reports - Research
Descriptors: Natural Language Processing, Profiles, Individual Differences, Semantics, Artificial Intelligence, Readability, Reading Skills, Science Instruction, Computer Software, Textbooks, Correlation, Reading Comprehension, Cues, Connected Discourse, Computational Linguistics, Instructional Material Evaluation
DOI: 10.20944/preprints202504.2426.v1
Abstract: We conducted two experiments to assess the alignment between Generative AI (GenAI) text personalization and hypothetical readers' profiles. In Experiment 1, four LLMs (i.e., Claude 3.5 Sonnet; Llama; Gemini Pro 1.5; ChatGPT 4) were prompted to tailor 10 science texts (i.e., biology, chemistry, physics) to accommodate four different profiles varying in knowledge, reading skills, and learning goals. Natural Language Processing (NLP) was leveraged to evaluate the GenAI adapted texts using an array of linguistic and semantic features empirically associated with text readability. NLP analyses revealed variations in the degree to which the LLMs successfully adjusted linguistic features to suit reader profiles. Most notably, NLP highlighted inconsistent alignment between potential reader abilities and text complexity. The results pointed toward the need to augment the AI prompts using personification, chain-of-thought, and documents regarding text comprehension, text readability and individual differences (i.e., leveraging RAG). The resulting text modifications in Experiment 2 were better aligned with readers' profiles. Augmented prompts resulted in LLM modifications with more appropriate cohesion features tailored to high and low knowledge readers for optimal comprehension. This study demonstrates how LLMs can be prompted to modify text and uniquely demonstrates the application of NLP to evaluate theory-driven content personalization using GenAI. NLP offers an efficient, real-time solution to validate personalized content across multiple domains and contexts. [Note: This content is a pre-print version of the article.]
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2025
Accession Number: ED673018
Database: ERIC
Description
Abstract:We conducted two experiments to assess the alignment between Generative AI (GenAI) text personalization and hypothetical readers' profiles. In Experiment 1, four LLMs (i.e., Claude 3.5 Sonnet; Llama; Gemini Pro 1.5; ChatGPT 4) were prompted to tailor 10 science texts (i.e., biology, chemistry, physics) to accommodate four different profiles varying in knowledge, reading skills, and learning goals. Natural Language Processing (NLP) was leveraged to evaluate the GenAI adapted texts using an array of linguistic and semantic features empirically associated with text readability. NLP analyses revealed variations in the degree to which the LLMs successfully adjusted linguistic features to suit reader profiles. Most notably, NLP highlighted inconsistent alignment between potential reader abilities and text complexity. The results pointed toward the need to augment the AI prompts using personification, chain-of-thought, and documents regarding text comprehension, text readability and individual differences (i.e., leveraging RAG). The resulting text modifications in Experiment 2 were better aligned with readers' profiles. Augmented prompts resulted in LLM modifications with more appropriate cohesion features tailored to high and low knowledge readers for optimal comprehension. This study demonstrates how LLMs can be prompted to modify text and uniquely demonstrates the application of NLP to evaluate theory-driven content personalization using GenAI. NLP offers an efficient, real-time solution to validate personalized content across multiple domains and contexts. [Note: This content is a pre-print version of the article.]
DOI:10.20944/preprints202504.2426.v1