NLP Validation of Prompt Strategies for Theory-Aligned LLM-Generated Personalization
Saved in:
| Title: | NLP Validation of Prompt Strategies for Theory-Aligned LLM-Generated Personalization |
|---|---|
| Language: | English |
| Authors: | Linh Huynh (ORCID |
| Source: | Grantee Submission. 2026. |
| Peer Reviewed: | Y |
| Page Count: | 9 |
| Publication Date: | 2026 |
| Sponsoring Agency: | National Center for Education Research (NCER) (ED/IES) |
| Contract Number: | R305T240035 |
| Document Type: | Speeches/Meeting Papers Reports - Research |
| Descriptors: | Natural Language Processing, Artificial Intelligence, Validity, Individualized Instruction, History Instruction, Reading Skills, Prior Learning, Educational Technology, Computer Uses in Education |
| DOI: | 10.59668/2551.25404 |
| Abstract: | This study applies an NLP-based validation framework to examine how Large Language Models (LLMs) can be iteratively refined for theory-aligned text personalization. Building on prior work, we extend the evaluation method to history texts and focus on prompt design as a key factor in personalization quality. Four LLMs (Claude, Llama, Gemini, ChatGPT-4) were prompted to adapt ten history passages for four reader profiles varying in reading skill and prior knowledge using one-shot and task-specific instruction prompts. Linguistic indices were extracted using the Writing Analytics Tool to assess the alignment of linguistic features with students' needs. Although LLMs appropriately tailored text complexity, cohesion patterns failed to match theoretical expectations even under explicit guidance. This iteration highlights the limits of current prompting strategies and the importance of theory-augmented refinement. Through iterative prompt evaluation, the study demonstrates how NLP provides a scalable, real-time framework for validating and improving theory-driven personalization across multiple domains. [This paper was published in: "Proceedings of the Learning Engineering Research Network Convening (LERN 2026)," 2026, pp. 278-285.] |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2026 |
| Accession Number: | ED678903 |
| Database: | ERIC |
| Abstract: | This study applies an NLP-based validation framework to examine how Large Language Models (LLMs) can be iteratively refined for theory-aligned text personalization. Building on prior work, we extend the evaluation method to history texts and focus on prompt design as a key factor in personalization quality. Four LLMs (Claude, Llama, Gemini, ChatGPT-4) were prompted to adapt ten history passages for four reader profiles varying in reading skill and prior knowledge using one-shot and task-specific instruction prompts. Linguistic indices were extracted using the Writing Analytics Tool to assess the alignment of linguistic features with students' needs. Although LLMs appropriately tailored text complexity, cohesion patterns failed to match theoretical expectations even under explicit guidance. This iteration highlights the limits of current prompting strategies and the importance of theory-augmented refinement. Through iterative prompt evaluation, the study demonstrates how NLP provides a scalable, real-time framework for validating and improving theory-driven personalization across multiple domains. [This paper was published in: "Proceedings of the Learning Engineering Research Network Convening (LERN 2026)," 2026, pp. 278-285.] |
|---|---|
| DOI: | 10.59668/2551.25404 |
