View in EDS HTML Full Text PDF Full Text

The Costs and Impacts of 'Descubriendo la Lectura': Evidence from a Multisite Experimental Study

Saved in:
Bibliographic Details
Title: The Costs and Impacts of 'Descubriendo la Lectura': Evidence from a Multisite Experimental Study
Language: English
Authors: Geoffrey D. Borman (ORCID 0000-0002-7039-8208), Iliana Brodziak de los Reyes, Trisha H. Borman, Scott Houghton, So Jung Park, Bo Zhu, Alejandra Martin
Source: Journal of Research on Educational Effectiveness. 2025 18(3):739-768.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 30
Publication Date: 2025
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R305A160060
Document Type: Journal Articles
Reports - Research
Education Level: Elementary Education
Early Childhood Education
Grade 1
Primary Education
Descriptors: Elementary Schools, Grade 1, Intervention, English Learners, Spanish Speaking, Literacy, Reading Achievement, Reading Difficulties, Program Implementation, Expenditure per Student, Educational Finance, Program Costs, School Districts
Geographic Terms: Arizona, California, Illinois, Texas, Wisconsin
DOI: 10.1080/19345747.2024.2358824
ISSN: 1934-5747
1934-5739
Abstract: We estimate costs and impacts of "Descubriendo la Lectura" (DLL), an intervention designed to improve the literacy skills of Spanish-speaking first graders struggling with reading. Collecting cost data from 24 schools in seven districts across four states participating in a multisite randomized controlled trial of DLL, we used the ingredients method to identify all personnel and nonpersonnel resources and then assign costs to each ingredient. The average cost per student of DLL is approximately $7,120 (in 2022 dollars), with teacher and teacher leader personnel expenditures accounting for more than 90% of the total costs. Impact analyses from the student-level random assignment study suggest DLL has substantial effects on Spanish-language literacy achievement, with effect sizes from d = 0.23 to d = 0.91. Linking estimated costs and achievement impacts, the impact per $1,000 investment in DLL is between d = 0.03 and d = 0.13, and a cost between $7,824 and $30,957 per student is associated with a one standard deviation increase in student achievement. These estimates compare favorably with those found for other interventions with recent cost and impact data.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2026
Accession Number: EJ1502025
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwEy00gdrPxlk8u6Zm4k9PE_AAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDEAfET_I8NmOq98MPgIBEICBmq4oR-WKXsQOjFVqXsablUDnkT20FlsW6eHaaaTu-VKKZfrpfrs8CWOJmBeHp6EgrhmblHQy5fK4BfgGSdWp9i76Unj2s2twR2w9AFNWfYh5NWBz_Fma3JjJMZUwDbwlMqIpJHkxCR4SZStUTBuTbd5Oe3ZMfEbh0rVbrITbIaI99s72qX0bWOXqBgnFKsPQ8L9pVIZxpowpudU=
Text:
  Availability: 1
  Value: <anid>AN0187023105;[5ew9]01jul.25;2025Aug01.02:50;v2.2.500</anid> <title id="AN0187023105-1">The Costs and Impacts of Descubriendo la Lectura: Evidence From a Multisite Experimental Study </title> <p>We estimate costs and impacts of Descubriendo la Lectura (DLL), an intervention designed to improve the literacy skills of Spanish-speaking first graders struggling with reading. Collecting cost data from 24 schools in seven districts across four states participating in a multisite randomized controlled trial of DLL, we used the ingredients method to identify all personnel and nonpersonnel resources and then assign costs to each ingredient. The average cost per student of DLL is approximately $7,120 (in 2022 dollars), with teacher and teacher leader personnel expenditures accounting for more than 90% of the total costs. Impact analyses from the student-level random assignment study suggest DLL has substantial effects on Spanish-language literacy achievement, with effect sizes from d = 0.23 to d = 0.91. Linking estimated costs and achievement impacts, the impact per $1,000 investment in DLL is between d = 0.03 and d = 0.13, and a cost between $7,824 and $30,957 per student is associated with a one standard deviation increase in student achievement. These estimates compare favorably with those found for other interventions with recent cost and impact data.</p> <p>Keywords: Cost-effectiveness; English learners; descubriendo la lectura; spanishlanguage literacy; randomized controlled trial</p> <p>The United States allocates considerable resources to public elementary and secondary education, spending $739 billion annually according to recent national data (National Center for Education Statistics, [<reflink idref="bib45" id="ref1">45</reflink>]). U.S. policymakers, practitioners, and researchers, however, remain impoverished with respect to the evidence needed to help understand the relative costs and impacts of educational programs and policies and their alternatives. In the 1960s, a small number of education policy and economics researchers began applying cost-effectiveness and benefit-cost methods to help guide decision making regarding the adoption of competing education programs and policies (Levin, [<reflink idref="bib35" id="ref2">35</reflink>]). However, in the decades since, few rigorous cost studies have been conducted, and little systematic evidence of the relative costs and impacts of replicable educational programs or policies has accumulated (Levin & Belfield, [<reflink idref="bib36" id="ref3">36</reflink>]). In recent years, the tide appears to be turning (Sparks, [<reflink idref="bib58" id="ref4">58</reflink>]). Under the leadership of Director Mark Schneider, the Institute of Education Sciences (IES) of the U.S. Department of Education recently adopted the Standards for Excellence in Education Research (SEER).[<reflink idref="bib1" id="ref5">1</reflink>] These standards augment the long-standing What Works Clearinghouse (WWC) guidelines on the internal validity of cause-and-effect evaluation studies (WWC, [<reflink idref="bib66" id="ref6">66</reflink>]), providing guidance for the field when conducting rigorous education research on "what works" that is transparent, actionable, and ultimately transformative. One key standard advanced by SEER is that researchers should engage in a cost study, which complements and extends the practical utility of an internally valid evaluation by documenting for policymakers, practitioners, and the research community the resources needed to implement the evaluated program or policy.</p> <p>As Belfield and Bowden ([<reflink idref="bib5" id="ref7">5</reflink>]) suggest, cost-effectiveness analyses rely on impact evaluations. In most studies of educational interventions, though, cost and impact evaluations are performed separately. Yet, as Belfield and Bowden noted, educational evaluations can be enhanced if education researchers and economists work together to integrate impact and cost analyses more directly. Working together as a research team to document impacts, implementation, and cost, we provide a rigorous and coherent study of <emph>Descubriendo la Lectura</emph> (DLL), an intervention designed to improve the literacy skills of Spanish-speaking first graders who struggle with reading. DLL is the Spanish-language version of the widely used Reading Recovery (RR) program.[<reflink idref="bib2" id="ref8">2</reflink>] Both interventions employ highly trained teachers who apply well-specified guidelines and standards to work one-on-one with students. DLL teachers receive support from a network of teacher leaders, who provide direct supervision, assistance, and professional development, and a group of university-based trainers, who provide initial and follow-up DLL training. The goal of the DLL and RR programs is to equip students with the strategies they need to improve their literacy skills and allow students to "catch up to their peers and continue to work on their own within an average group setting in the regular program" (RRCNA, [<reflink idref="bib48" id="ref9">48</reflink>], p. 1).</p> <hd id="AN0187023105-2">The DLL Program</hd> <p>A growing literature indicates that Spanish-speaking English learners (ELs) benefit from early literacy instruction offered in their first language. Nevertheless, some children afforded this opportunity will continue to struggle to acquire essential early literacy skills—even in their native or first language. In these cases, children need interventions to help them develop Spanish literacy skills so that they can become successful readers in their first language and can then, over time, apply and transfer these skills to their English literacy development. With 80% of ELs claiming Spanish as their home language (Calderón et al., [<reflink idref="bib11" id="ref10">11</reflink>]) and with the continued growth of bilingual programs across the United States, the number of students receiving their initial literacy instruction in Spanish will increase, as will the need for a Spanish-language early intervention program for students at risk of literacy failure. One such program is DLL (RRCNA, [<reflink idref="bib48" id="ref11">48</reflink>]).</p> <p>DLL provides one-on-one Spanish-language tutoring for first-grade Spanish-speaking students who experience substantial challenges with reading and writing. DLL extends the RR approach (Sirinides et al., [<reflink idref="bib54" id="ref12">54</reflink>]) to ELs who struggle with early literacy by first addressing their learning needs in their native Spanish language. The program aims to improve early reading and writing outcomes for the lowest achieving ELs through a focused, short-term approach that offers the needed boost to catch up to their classmates and, ultimately, achieve their long-term academic outcomes (RRCNA, [<reflink idref="bib48" id="ref13">48</reflink>]). DLL offers daily 30-minute instruction for 12–20 weeks for the lowest performing students (RRCNA, [<reflink idref="bib48" id="ref14">48</reflink>]). Trained literacy teachers provide intensive, individualized support to complement students' regular-classroom instruction, with the goal of raising their literacy skills to the level of their peers (RRCNA, [<reflink idref="bib48" id="ref15">48</reflink>]).</p> <p>The practice of providing first-language instruction and support to struggling Spanish-speaking ELs has considerable empirical support. Recent studies comparing bilingual and English-only programs tend to indicate that bilingual literacy instruction produces stronger long-term English proficiency and English language arts achievement (NASEM, [<reflink idref="bib43" id="ref16">43</reflink>]; Umansky & Reardon, [<reflink idref="bib60" id="ref17">60</reflink>]; Valentino & Reardon, [<reflink idref="bib62" id="ref18">62</reflink>]). Further, six research syntheses suggest that, compared with immersing students in English, teaching students in both their native language and in English produces superior literacy results overall (Francis et al., [<reflink idref="bib21" id="ref19">21</reflink>]; Greene, [<reflink idref="bib24" id="ref20">24</reflink>]; McField & McField, [<reflink idref="bib42" id="ref21">42</reflink>]; Rolstad et al., [<reflink idref="bib51" id="ref22">51</reflink>]; Slavin & Cheung, [<reflink idref="bib56" id="ref23">56</reflink>]; Willig, [<reflink idref="bib67" id="ref24">67</reflink>]).</p> <p>Beyond Spanish-language instruction, individualized, tailored instruction is one of the defining features of DLL. Extensive training prepares teachers to offer moment-to-moment responses to student behaviors and to diagnose and respond to individual student's needs (RRCNA, [<reflink idref="bib48" id="ref25">48</reflink>]). Teaching decisions depend on careful observation and an understanding of student progress; thus, program implementation relies on and demands extensive record-keeping. Teachers must maintain records for each student—using data to inform their practice—by employing a tool that captures literacy progress before entering the program, upon leaving the program, and at the end of the school year: the <emph>Instrumento de Observación</emph> (<emph>IdO</emph>; RRCNA, [<reflink idref="bib48" id="ref26">48</reflink>]). Teachers also must use a variety of tools to routinely assess student progress, including a planning report, daily monitoring tools, and daily assessment forms.</p> <p>DLL strives for a sustainable and scalable system that prioritizes implementation of high-quality instruction with fidelity. It accomplishes this fidelity of implementation through continuous, district-level monitoring, national standards and guidelines, and feedback provided by teacher leaders to key DLL staff (e.g., administrators, teachers; May et al., [<reflink idref="bib41" id="ref27">41</reflink>]). The program model further supports quality instruction through stringent teacher selection standards, extensive training, and job-embedded ongoing professional development. University-trained teacher leaders provide a year of intensive training in Spanish or a year of English training and a second year of "bridging" to Spanish. Following this training, teacher leaders provide close supervision during the teacher's field year and, in future years, multiple professional development sessions and continuous technical assistance, including one-on-one coaching. These high standards and ongoing mentoring opportunities combine to produce teacher expertise and, ultimately, reflect the principle that instructional quality is paramount (May et al., [<reflink idref="bib41" id="ref28">41</reflink>]).</p> <hd id="AN0187023105-3">An Introduction to the DLL Program Ingredients</hd> <p>By taking a detailed and systemic approach to account for all resources needed to implement the program and their costs, we can provide best estimates of the resources needed to replicate DLL in another location. We followed the ingredients approach to cost analysis to identify all personnel and nonpersonnel resources associated with the implementation of the DLL program in the sampled schools (Levin et al., [<reflink idref="bib37" id="ref29">37</reflink>]). Based on this approach, we gathered data to account for all resources used in implementing the program and classified the DLL ingredients into the following categories: personnel, facilities, supplies and equipment, and other inputs. Detailed descriptions of these ingredients are in the following subsections.</p> <hd id="AN0187023105-4">Personnel</hd> <p>As with the field of education in general (National Center for Education Statistics, [<reflink idref="bib45" id="ref30">45</reflink>]), DLL is a personnel-intensive program, and the vast majority of resources are spent on teachers and other staff. DLL employs teachers to provide one-on-one tutoring to up to eight students per academic year. With fidelity of implementation a clear priority, DLL depends on continuous monitoring and feedback, as provided by teacher leaders and teacher trainers. The teacher leaders are selected by a district or a consortium of districts that makes a commitment to implement DLL. They teach children, train DLL teachers for local schools, provide ongoing support and feedback to DLL teachers, analyze and report student outcomes, and communicate with local educators regarding the coordination of DLL with other school and district policies and practices. Teacher trainers at the regional university training centers are typically university-based faculty employed to provide initial and ongoing professional development for DLL teacher leaders and teachers. They also support implementation within their network of affiliated sites, help develop new implementations, conduct research, and generally maintain the integrity of DLL within their region. Finally, principals and other educators at participating schools spend time coordinating and collaborating with DLL teachers.</p> <p>Teachers, teacher leaders, and teacher trainers participate in a set of yearly professional development activities. Teachers participate in at least six professional learning sessions each year, with a minimum of four sessions providing opportunities to observe and discuss live teaching sessions. Teachers also attend at least one conference per year and collaborate with other teachers, administrators, and school teams in their own schools. Ongoing professional development for teacher leaders helps them remain knowledgeable about current DLL best practices. Their training emphasizes acquiring skills and knowledge to facilitate the growth and development of teachers, evaluate and communicate the effectiveness of DLL teaching, and offer guidance for implementation decisions in and across schools. The network of university-based trainers also convenes for extended professional development at least twice each year. The university-based trainers, along with other academics and educational leaders outside DLL, work together to maintain up-to-date perspectives on research and practice.</p> <hd id="AN0187023105-5">Facilities</hd> <p>The DLL program is typically delivered within the school that a participating child attends. A quiet dedicated space within each school is necessary to deliver one-to-one tutoring. Although DLL does not demand significant investments in facilities, it does require one particularly unique facility: access to a space with a one-way glass installed between a large classroom and a small tutoring room. As part of the DLL training and professional development model, teachers must deliver tutoring sessions periodically while being observed and evaluated by their teacher leader and other DLL personnel through the one-way glass wall. The large classroom on the other side of the wall allows other DLL teachers and teacher leaders—and, in some cases, the principal, other first-grade teachers, or administrators considering implementing DLL—to unobtrusively observe the lessons. In addition to the one-way glass observation room, the teacher leader typically has an adjacent workspace at the district or regional office.</p> <hd id="AN0187023105-6">Supplies and Equipment</hd> <p>DLL requires a supply of new and engaging books for the participating students, testing materials for monitoring student progress, and supporting textbooks for teachers and teacher leaders. Beyond these essential supplies, DLL teachers and teacher leaders require computer equipment to document and monitor student progress and support their work.</p> <hd id="AN0187023105-7">Other Inputs</hd> <p>Finally, the focus of DLL on continuous professional development includes the spending associated with DLL trainings, such as fees; tuition; and transportation to and from courses, conferences, and other professional development opportunities. Districts also pay a fee per school and per teacher to the International Data Evaluation Center (IDEC) that supports data management.</p> <hd id="AN0187023105-8">The Current Study</hd> <p>With this study, we present new evidence of costs and cost-effectiveness of implementing the DLL program with EL first graders enrolled in public schools. The full randomized control trial (RCT) of DLL was conducted over three academic years (2016–17, 2017–18, and 2018–19), with the cost study drawing on observations from the 2017–18 school year. The cost study was guided by the following research questions:</p> <p></p> <ulist> <item> What is the total cost and the average annual cost per student to districts that implement the DLL program?</item> <p></p> <item> What are the startup (i.e., onetime) and recurring costs to districts that implement the DLL program?</item> <p></p> <item> How are program costs distributed across the types of resources that schools use to implement the program?</item> <p></p> <item> What is the cost-effectiveness ratio of DLL relative to "business-as-usual" literacy instruction?</item> <p></p> <item> What is DLL's relative cost-effectiveness compared to other programs schools might use to improve the Spanish-language literacy skills of young Spanish-speaking students?</item> </ulist> <p>A resource cost model (RCM) framework was used to estimate the cost of implementing DLL at the participating sites. RCM applies an economic lens to generate estimates of educational program costs by identifying all resources (e.g., staff time) used to provide a service or program from the "bottom up," subsequently assigning dollar values to these resources (Chambers, [<reflink idref="bib12" id="ref31">12</reflink>]). RCM stands in contrast to "top-down" accounting-oriented approaches used to identify and value program resources, such as budgets or expenditure data (Hartman et al., [<reflink idref="bib26" id="ref32">26</reflink>]), that provide a mechanism for how dollars are spent but do not provide the types of information required to understand what resources were used by a program and would be necessary for program replication. In this study, we link cost estimates with multiple measures of DLL program impacts to generate ratios of costs and effects. We then compare DLL cost-effectiveness ratios (CERs) to those of other comparable programs to inform future decisions to adopt DLL or other programs for struggling early bilingual readers.</p> <hd id="AN0187023105-9">Method</hd> <p></p> <hd id="AN0187023105-10">Sample</hd> <p>Table 1 describes the samples for the cost and impact studies. The targeted sample for the 2017–18 cost and impact studies included the same 27 schools from 10 districts across five states.[<reflink idref="bib3" id="ref33">3</reflink>] Three schools declined to participate in the cost study, so the analytic sample for the cost study included 24 schools from seven districts across four states. Nineteen DLL teachers and eight DLL teacher leaders serving 199 students participated in the cost study. To calculate the DLL CER, we linked cost data with our estimates of student-level achievement impacts for the same school year (2017–18). The final state, district, teacher leader, DLL teacher, and student samples included considerable overlap between the cost and impact studies. Table 1 also reveals where cost data were unavailable because of nonparticipation.</p> <p>Table 1. Summary of cost and impact study samples.</p> <p> <ephtml> <table><thead><tr><td /><td>Teacher leaders (<italic>N</italic>)</td><td>Schools (<italic>N</italic>)</td><td>Teachers (<italic>N</italic>)</td><td>Students (<italic>N</italic>)</td></tr><tr><td>State</td><td>2017–18 district locale</td><td>Impact study</td><td>Cost study (sampled)</td><td>Cost study (responded)</td><td>Impact study</td><td>Cost study</td><td>Impact study</td><td>Cost study (sampled)</td><td>Cost study (responded)</td><td>Impact study</td><td>Cost study<xref ref-type="table-fn" rid="tfn2">a</xref></td></tr></thead><tbody valign="top"><tr><td>Arizona</td><td>City: large</td><td char=".">1</td><td /><td /><td char=".">1</td><td /><td char=".">2</td><td /><td /><td char=".">4</td><td /></tr><tr><td>California</td><td>Suburb: large</td><td char=".">1</td><td /><td /><td char=".">1</td><td /><td char=".">1</td><td /><td /><td char=".">8</td><td /></tr><tr><td>California</td><td>City: small</td><td char=".">1</td><td char=".">1</td><td char=".">1</td><td char=".">1</td><td char=".">1</td><td char=".">1</td><td char=".">3</td><td char=".">1</td><td char=".">8</td><td char=".">30</td></tr><tr><td>California</td><td>Suburb: large</td><td char=".">1</td><td /><td /><td char=".">1</td><td /><td char=".">1</td><td /><td /><td char=".">8</td><td /></tr><tr><td>Illinois</td><td>City: small</td><td char=".">2</td><td char=".">2</td><td char=".">2</td><td char=".">5</td><td char=".">5</td><td char=".">5</td><td char=".">5</td><td char=".">5</td><td char=".">29</td><td char=".">38</td></tr><tr><td>Illinois</td><td>Suburb: large</td><td char=".">1</td><td char=".">1</td><td char=".">1</td><td char=".">2</td><td char=".">2</td><td char=".">2</td><td char=".">2</td><td char=".">2</td><td char=".">10</td><td char=".">10</td></tr><tr><td>Texas</td><td>City: midsize</td><td char=".">2</td><td char=".">2</td><td char=".">1</td><td char=".">3</td><td char=".">3</td><td char=".">3</td><td char=".">4</td><td char=".">1</td><td char=".">24</td><td char=".">34</td></tr><tr><td>Texas</td><td>City: midsize</td><td char=".">1</td><td char=".">1</td><td char=".">1</td><td char=".">3</td><td char=".">3</td><td char=".">4</td><td char=".">4</td><td char=".">3</td><td char=".">19</td><td char=".">24</td></tr><tr><td>Texas</td><td>Suburb: large</td><td char=".">1</td><td char=".">1</td><td char=".">1</td><td char=".">7</td><td char=".">7</td><td char=".">7</td><td char=".">7</td><td char=".">4</td><td char=".">39</td><td char=".">45</td></tr><tr><td>Wisconsin</td><td>City: small</td><td char=".">1</td><td char=".">1</td><td char=".">1</td><td char=".">3</td><td char=".">3</td><td char=".">3</td><td char=".">3</td><td char=".">3</td><td char=".">17</td><td char=".">18</td></tr><tr><td>Sample totals</td><td char=".">12</td><td char=".">9</td><td char=".">8</td><td char=".">27</td><td char=".">24</td><td char=".">29</td><td char=".">28</td><td char=".">19</td><td char=".">166</td><td char=".">199</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note.</emph> We used the locale codes provided by the National Center for Education Statistics ([<reflink idref="bib44" id="ref34">44</reflink>]) to describe a district's location, ranging from "large city" to "rural." The number of students for the cost study was based on data from the International Data Evaluation Center (IDEC).</p> <p>2 The cost study sample represents the total number of students who received the <emph>Descubriendo la Lectura</emph> (DLL) intervention during the 2017–18 academic year (<emph>n</emph> = 199). This count of students differs from the number of students in the impact study sample for the same academic year (<emph>n</emph> = 166) for two reasons: (a) Three districts that participated in the impact study did not participate in the cost study; and (b) in participating cost study sites, the count of students receiving DLL that districts reported to IDEC was larger than the number of students randomly assigned to the intervention at the start of the school year. This latter condition is because some students were identified for DLL after random assignment occurred. Given that providing services to these students was part of a district's costs to implement the program, the additional students (<emph>n</emph> = 33) are considered in the cost analysis.</p> <hd id="AN0187023105-11">Cost Analysis</hd> <p>We used the ingredients method to identify program resources and construct estimates of the economic costs associated with program implementation at selected school sites (Levin et al., [<reflink idref="bib37" id="ref35">37</reflink>]). This method calls for enumerating all the resources (i.e., ingredients) used by a program to produce observed effects. We collected data from a variety of sources, including (a) documents that described the program and its implementation at selected sites; (b) interviews with DLL university trainers and teacher leaders; (c) cost questionnaires administered to DLL teachers and teacher leaders; and (d) information on the number of lessons delivered and the number of DLL students per teacher, as reported by teachers to the IDEC. When possible, we triangulated information about program resources among sources (e.g., IDEC student-level information regarding the number of sessions and number of weeks with information reported on teacher surveys). The following subsections describe in more detail each data source. Table 2 describes the types and amounts of key resources used to implement the DLL program along with the primary source of data regarding each intervention characteristic.</p> <p>Table 2. DLL intervention characteristics for participating districts (2017–18 School Year).</p> <p> <ephtml> <table><thead><tr><td>Characteristic</td><td>Intervention characteristics</td><td>Source</td></tr><tr><td>Minimum</td><td>Maximum</td><td>Mean</td></tr></thead><tbody valign="top"><tr><td>Number of DLL students served per teacher during the academic year</td><td>2 DLL students</td><td>16 DLL students</td><td>7 DLL students</td><td>IDEC data</td></tr><tr><td>Number of weeks that DLL intervention sessions were conducted during the academic year per teacher</td><td>11 weeks</td><td>20 weeks</td><td>15 weeks</td><td>IDEC data</td></tr><tr><td>Number of DLL sessions conducted during the academic year per teacher</td><td>33 sessions</td><td>68 sessions</td><td>51 sessions</td><td>IDEC data</td></tr><tr><td>Time spent per DLL intervention session with an individual student during the academic year</td><td>30 minutes</td><td>35 minutes</td><td>31 minutes</td><td>Teacher cost questionnaire</td></tr><tr><td>Time spent preparing for an individual DLL session during the academic year</td><td>10 minutes</td><td>45 minutes</td><td>25 minutes</td><td>Teacher cost questionnaire</td></tr><tr><td>Number of DLL intervention sessions conducted by a teacher per week during the academic year</td><td>2 sessions per week</td><td>20 sessions per week</td><td>9 sessions per week</td><td>Teacher cost questionnaire</td></tr></tbody></table> </ephtml> </p> <p>3 <emph>Note</emph>. DLL: Descubriendo la Lectura; IDEC: International Data Evaluation Center.</p> <hd id="AN0187023105-12">Document Reviews</hd> <p>We collected documents from DLL training centers that described the implementation of the DLL program at each site. Program documents identified key components, activities, and resources that the training centers used to implement the DLL program. The most important of these resources were the RR and DLL guidelines that described the requirements for implementing the DLL program. We also gathered documents that described the trainings and how much the training centers spent to provide DLL-related trainings. In addition, the documents provided information on what the training centers spent on nonpersonnel items (e.g., tuition, materials such as books for the training and the teacher's library). We used this information to inform our interview protocol and the cost questionnaires.</p> <hd id="AN0187023105-13">Interviews</hd> <p>We interviewed teacher leaders and DLL university trainers who had been involved with the DLL program for more than 1 year to better understand the tasks and activities that teacher leaders performed. Interviewees provided useful information about site-specific implementation of the program and teacher time commitments, as well as the fees related to the trainings and other professional development activities. Interviews with university trainers provided information on the organization of DLL conferences and teacher professional development (e.g., conference frequency and the resources used to train DLL teachers and teacher leaders; supplies and materials). Altogether, we conducted eight 60-minute interviews with teacher leaders and DLL university trainers from Texas Women's University and two university trainers, one from St. Mary's University and the other from the National Louis University. We used this information to inform the cost questionnaires.</p> <hd id="AN0187023105-14">Cost Questionnaires</hd> <p>We developed two online questionnaires to collect information from DLL teacher leaders and teachers about the time spent and other resources used to implement the program during the 2017–18 school year. The teacher questionnaire captured responses across the following categories: (a) professional development, (b) lesson preparation and program implementation, (c) data collection, (d) materials and facilities, (e) administrative tasks, and (f) communications related to implementation. In addition, the questionnaires included items about nonpersonnel resources (e.g., supplies, materials, facilities) involved in delivering the intervention. Respondents were asked to consider both the time and resources needed to start the DLL program in their school as well as operate the program on an ongoing basis. We pilot tested the cost questionnaires with two DLL teachers and two DLL teacher leaders and made revisions to the questionnaires based on feedback.</p> <p>The online questionnaires were administered to DLL teacher leaders and DLL teachers between February and April 2019. Questionnaires were completed by eight DLL teacher leaders (89%) and 19 DLL teachers (68%; see also Table 1). We imputed costs and hours for DLL teachers and teacher leaders who did not complete the cost questionnaire based on the median per-student allocations of all other DLL teachers and teacher leaders within each site.</p> <hd id="AN0187023105-15">IDEC Data System</hd> <p>Data from the IDEC system provided information on the actual number of students who received the DLL intervention in each district during the 2017–18 academic year and the number of sessions and weeks that teachers spent with each student. These data were cross-checked with information collected from DLL teachers using the cost questionnaires.</p> <hd id="AN0187023105-16">Determining Prices</hd> <p></p> <hd id="AN0187023105-17">Personnel Prices</hd> <p>The price of personnel time is equivalent to the compensation paid to staff, inclusive of wages and benefits. Annual salaries for DLL teachers were determined using each district's 2017–18 salary schedule, accounting for differences in teachers' years of experience and highest education degree. For other types of staff (e.g., principals, classroom teachers, specialists), we used the median annual salary for each job type reported by the U.S. Department of Labor's Bureau of Labor Statistics' Occupational Employment Statistics Survey for the metropolitan area in which the districts were located. The median annual salaries were adjusted for regional variations in the wages or salaries of college graduates using the U.S. Department of Education's 2018 Comparable Wage Index for Teachers (Cornman et al., [<reflink idref="bib18" id="ref36">18</reflink>]). We then calculated a benefit ratio for each state in which districts were located using information from the U.S. Department of Education's <emph>Revenue and Expenditures for Public Elementary and Secondary Education: FY18</emph> (National Center for Education Statistics, [<reflink idref="bib46" id="ref37">46</reflink>], Table 6). The statewide benefits ratio was multiplied by the geographically adjusted salary amounts for DLL teachers and other staff to determine total annual compensation for each personnel resource.</p> <p>Because personnel time devoted to DLL program activities was reported in hours, we converted total annual compensation for each personnel resource to an hourly rate. Using data from the National Center for Education Statistics' 2011-12 <emph>Schools and Staffing Survey</emph>, we divided total annual compensation by the typical number of hours that teachers and other school staff were required to work to receive their base pay. The hourly compensation rates were used to calculate the cost of personnel time.</p> <hd id="AN0187023105-18">Nonpersonnel Prices</hd> <p>Prices for supplies and equipment were collected in two ways. In some cases, staff provided exact prices in their interview responses. For example, some study participants reported prices for professional development (e.g., tuition) and the costs for textbooks. In other cases, prices for supplies and equipment were collected from national online retailers. Facilities prices came from two sources: (a) the <emph>CostOut<sups>®</sups></emph> database developed by the Center for Benefit-Cost Studies in Education at Teachers College, Columbia University (Hollands et al., [<reflink idref="bib29" id="ref38">29</reflink>]) and (b) actual spending by districts on facilities (e.g., one-way glass classroom). We used this information to cost out classroom space and the one-way glass rooms required for the DLL intervention. Our estimates for DLL costs include the time the facilities were used by the DLL program (see Appendix Table A1).</p> <hd id="AN0187023105-19">Calculating Cost</hd> <p>Using data from the cost questionnaires, interviews, and document reviews, we developed an RCM for each district to estimate total and average annual per student costs to implement DLL. Every resource used to implement DLL was entered in the RCM and assigned to an activity (e.g., initial training, ongoing professional development, DLL program delivery). Resources were further categorized according to ingredient type (personnel, facilities, supplies and equipment, and other inputs) and whether the resource was part of the DLL program startup or its ongoing operation. All ingredients also were categorized according to whether they were part of the DLL program startup or its ongoing operation.</p> <p>Cost estimates reflect personnel costs, including all staff time dedicated to DLL activities; facilities costs, including space used to implement the DLL intervention (e.g., meeting space to conduct trainings, classroom space to provide the services to children, and the one-way mirrored observation room); supplies and equipment costs include all children's books, textbooks, and computers used to support DLL implementation. Other inputs include tuition and fees for conferences and courses plus transportation costs to and from trainings and other professional development activities.</p> <p>The types, quantities, and prices for each resource used to implement DLL in a district were entered in the RCM. For resources shared with other activities, we considered only the amount a resource was used to implement the DLL intervention.[<reflink idref="bib4" id="ref39">4</reflink>] For resources with useful lifespans of more than 1 year, we also noted the number of years before the resource would need replacement or refreshing. We used this information to calculate the cost of each resource, <emph>r</emph>, used for each activity as</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mtext mathvariant="italic">Cost</mtext></mrow><mrow><mi>r</mi><mo>,</mo><mi>a</mi><mo>,</mo><mi>t</mi></mrow></msub></mrow><mo>=</mo><mrow><mfrac><mrow><mrow><msub><mrow><mtext mathvariant="italic">Quantity</mtext></mrow><mrow><mi>r</mi><mo>,</mo><mi>a</mi><mo>,</mo><mi>t</mi></mrow></msub></mrow><mo /><mi>X</mi><mo /><mrow><msub><mrow><mtext mathvariant="italic">Price</mtext></mrow><mrow><mi>r</mi><mo>,</mo><mi>a</mi><mo>,</mo><mi>t</mi></mrow></msub></mrow></mrow><mrow><mrow><msub><mrow><mtext mathvariant="italic">Lifespan</mtext></mrow><mrow><mi>r</mi><mo>,</mo><mi>a</mi><mo>,</mo><mi>t</mi></mrow></msub></mrow></mrow></mfrac></mrow></math> </ephtml> </p> <p>where <emph>Quantity</emph> is measured in full-time equivalents for personnel and general units for nonpersonnel, <emph>Price</emph> is measured as hourly compensation for personnel or a general price per unit for nonpersonnel, and <emph>Lifespan</emph> is the number of years before the resource will need to be replaced. For the <emph>Cost</emph>, <emph>Quantity</emph>, <emph>Price</emph>, and <emph>Lifespan</emph> of each resource, <emph>r</emph>, we use the subscripts <emph>a</emph> and <emph>t</emph> to denote the activity and resource type, respectively.</p> <p>An amortization period was combined with an interest rate to calculate the average annual costs of ingredients that are useful for multiple years.[<reflink idref="bib5" id="ref40">5</reflink>] Annualization was used for nonpersonnel resources and the initial training that DLL teachers and teacher leaders received. For DLL teacher training, we assumed that the average DLL teacher or teacher leader would stay in their role for 10 years and used this as the amortization period for initial training costs. Appendix Table A1 provides the amortization periods, expressed in years, for nonpersonnel resources.</p> <p>For each district, we calculated the total cost of implementing the DLL program for the 2017–18 school year. The total cost to a district is the sum of the cost for each identified program resource (personnel and nonpersonnel). We then adjusted total costs for inflation to represent 2022 dollars using the Consumer Price Index (U.S. Department of Labor, [<reflink idref="bib61" id="ref41">61</reflink>]). We also calculated an average cost per student who participated in the DLL program, for each district and overall. The average cost per student for a district was calculated as the total cost of implementing the DLL program divided by the number of students in that district who participated in the DLL program during the 2017–18 school year.[<reflink idref="bib6" id="ref42">6</reflink>] The average cost per student, by district, helps us better understand how implementation costs vary according to program size. Because we were interested in calculating a CER that reflects the experiences of all districts in the study, we also calculated an average annual cost per student across all sites. To do so, we summed the total cost of implementing the DLL program (for all districts) and divided this sum by the total number of students for all sites (<emph>n</emph> = 199) that participated in the DLL program during the 2017–18 school year.</p> <hd id="AN0187023105-20">Achievement Impact Estimates</hd> <p>We summarize the study design, sample, achievement measures, and analytical methods in this section to facilitate interpretation of the achievement impact estimates, which we used to calculate the CERs for DLL. We conducted three yearly impact studies during the 2016–17, 2017–18, and 2018–19 academic years. Prior results for the 2016–17 cohort are reported by Borman et al. ([<reflink idref="bib9" id="ref43">9</reflink>]), and the combined results across all three cohorts are reported by Borman et al. ([<reflink idref="bib10" id="ref44">10</reflink>]). Consistent with the cost study, we report only the corresponding impacts observed during the 2017–18 school year. We employed an RCT design in which the lowest performing native Spanish-speaking first-grade students were randomly assigned to immediate treatment or a delayed, poststudy treatment condition. Our study focused on the fall semester outcomes for the students and contrasts the pre-post Spanish-language literacy outcomes for the immediately treated students to those of students whose treatment was delayed until after study completion. Following typical DLL practices, during fall 2017–18, DLL teachers administered a Spanish literacy assessment (<emph>IdO</emph>) to all at-risk Spanish-speaking students who were identified by their classroom teachers as performing below grade level in literacy in their school. Based on the <emph>IdO</emph> results, the lowest performing students at each campus were identified and randomized to immediate DLL treatment or a poststudy, delayed DLL treatment condition. Those students randomized to the immediately treated condition received DLL services at the beginning of the academic year, and those in the delayed condition received services 12–20 weeks later—as immediately served students exited the program.</p> <p>The researchers achieved this randomized design by generating a random number for each eligible child (i.e., the lowest performing ELs) at the beginning of the school year. Teachers served those with lower randomly assigned numbers with first priority; those students with higher numbers remained on a waiting list and were served later, after the conclusion of the study. As each immediately treated student completed DLL, the teachers served the next child on the waiting list with the lowest randomly assigned number. As students assigned to immediate service completed DLL, they were tested using the instruments described in the next subsections. At the same time, prior to entry into DLL, the next child on the randomly determined waiting list also was administered the three instruments. Thus, all students—both immediate and delayed treatment—were administered the fall <emph>IdO</emph> and <emph>Logramos</emph> ([<reflink idref="bib38" id="ref45">38</reflink>]) baseline tests during August or September, prior to assignment to DLL, and all students were again tested, using the <emph>IdO</emph> and <emph>Logramos</emph> instruments, after 12–20 weeks during November through February.</p> <hd id="AN0187023105-21">Achievement Impact Sample</hd> <p>The 2017–18 impact study included a sample of nearly 170 first-grade struggling readers, whose home language was Spanish, nested within 27 schools and ten districts across five states.[<reflink idref="bib7" id="ref46">7</reflink>] Ninety-nine percent of the students reported their race/ethnicity as Latinx, 97% identified as eligible for free or reduced-price meals, and 36% identified as female. All sampled schools offered transitional bilingual programs, which provided Spanish-language regular-classroom reading and language arts instruction to their first-grade ELs. DLL teachers provided supplemental one-on-one Spanish-language literacy tutoring to four students each semester, and each child progressed through the program at an individualized pace, generally lasting between 12 and 20 weeks. All students were native Spanish speakers.</p> <hd id="AN0187023105-22">Achievement Measures</hd> <p>The student assessments that we focus on here are the <emph>IdO</emph> (Escamilla et al., [<reflink idref="bib20" id="ref47">20</reflink>]) and the <emph>Logramos</emph> ([<reflink idref="bib38" id="ref48">38</reflink>]) reading achievement test. The <emph>IdO</emph> is the Spanish-language version of the <emph>Observation Survey of Early Literacy Achievement</emph> (<emph>OS</emph>; Clay, [<reflink idref="bib15" id="ref49">15</reflink>]). The <emph>OS</emph> and the <emph>IdO</emph> measure student performance on the following six subtests: Letter Identification, Ohio Word Test, Concepts About Print, Writing Vocabulary, Dictation, and Text Reading (Clay, [<reflink idref="bib16" id="ref50">16</reflink>]). Both the <emph>OS</emph> (for English-language students) and the <emph>IdO</emph> (for Spanish-language students) are administered as a regular component of the RR and DLL programs, respectively. The <emph>OS</emph> and the <emph>IdO</emph> tests are administered at the beginning of the first-grade school year as a screener to determine eligibility for RR and DLL services and after students complete services to measure academic progress. Both the <emph>OS</emph> and <emph>IdO</emph> are tools designed for the systematic observation of young children's early literacy competencies. The National Center on Response to Intervention reviewed <emph>OS</emph>, giving it its highest rating of "convincing evidence" in each category.[<reflink idref="bib8" id="ref51">8</reflink>]</p> <p> <emph>Logramos</emph> is the Spanish-language version of the <emph>Iowa Test of Basic Skills</emph> (ITBS) achievement test battery and is available from Houghton Mifflin Harcourt (<emph>Logramos</emph>, [<reflink idref="bib38" id="ref52">38</reflink>]). The items follow the scope and sequence of Form E of the English-language ITBS and have been translated, when appropriate, from the original English items in Form E of the ITBS. Although the vast majority of items were direct translations of the English test into Spanish, some items required adaptation or replacement of English items in the Spanish version to target the same skills and maintain the underlying psychometrics of the test items.</p> <p>For the <emph>Logramos</emph>, we administered three subtests during each assessment—Vocabulary/Vocabulario, Reading/Lectura, and Language/Lenguaje—which are combined to create an overall total score. Vocabulary/Vocabulario includes 26 items. Reading/Lectura includes 35 items and assesses literacy in the following domains: Literary Text/Texto literario, Informational Text/Texto informative, Explicit Meaning/Significado explícito, Implicit Meaning/Significado implícito, and Key Ideas/Ideas principals. The Language/Lenguaje section of the assessment includes 34 items assessing Spelling/Ortografía, Capitalization/Uso de mayúsculas, Punctuation/Puntuación, and Written Expression/Expresión escrita. <emph>Logramos</emph> is a broad-based, standardized, norm-referenced test that is not overaligned with the intervention; therefore, it provides a fair assessment of achievement change for both treatment- and control-group students. The <emph>Logramos</emph> was administered as an additional assessment for this the study; it is not typically given to DLL students under routine, nonstudy conditions. Both the individually administered <emph>IdO</emph> and group-administered <emph>Logramos</emph> posttests were proctored by a DLL teacher leader or by another teacher who did not provide instruction to the tested students.</p> <hd id="AN0187023105-23">Analytical Methods for Estimating Achievement Outcomes</hd> <p>We fit a series of hierarchical linear models to estimate the student-level "intent to treat" impact, or the impact of being assigned to receive DLL services, on each outcome measure. The unit of analysis is at the student level because students were randomly assigned within schools in this study. To account for the nested structure of data (in which students were nested within schools), the model includes a random error term for both the intercept and the estimate of the DLL impact, or "DLL slope," at the school level. The model adjusts outcomes by using a student-level pretest measure, thus improving model fit and statistically adjusting for chance pretest differences between the randomly assigned treatment and control groups. The model used for the overall impact analysis is</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mi>Y</mi></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub></mrow><mo>=</mo><mi mathvariant="normal" /><mi>α</mi><mo>+</mo><mi mathvariant="normal" /><mrow><msub><mrow><mi>β</mi></mrow><mrow><mn>1</mn></mrow></msub></mrow><mo stretchy="true">(</mo><mrow><mrow><msub><mrow><mtext mathvariant="italic">DLL</mtext></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub></mrow></mrow><mo stretchy="true">)</mo><mo>+</mo><mi mathvariant="normal" /><mrow><msub><mrow><mi>β</mi></mrow><mrow><mn>2</mn></mrow></msub></mrow><mo stretchy="true">(</mo><mrow><mrow><msub><mrow><mtext mathvariant="italic">PRETEST</mtext></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub></mrow></mrow><mo stretchy="true">)</mo><mo>+</mo><mi mathvariant="normal" /><mrow><msub><mrow><mi>u</mi></mrow><mrow><mi>j</mi></mrow></msub></mrow><mo>+</mo><mi mathvariant="normal" /><mrow><msub><mrow><mi>ε</mi></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub></mrow><mo>,</mo></math> </ephtml> </p> <p>where</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mi>Y</mi></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub></mrow><mo /></math> </ephtml> represents the test score of student <emph>i</emph> in school <emph>j</emph>, <emph>DLL<subs>ij</subs></emph> is an indicator variable equal to 1 for DLL (immediately treated group) and 0 for the control group (delayed treatment group), and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mo /><mi>β</mi></mrow><mrow><mn>1</mn></mrow></msub></mrow><mo /></math> </ephtml> is the coefficient representing the impact of DLL for student <emph>i</emph> in school <emph>j</emph>. The term</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mi>β</mi></mrow><mrow><mn>2</mn></mrow></msub></mrow></math> </ephtml> represents the student-level association between the pretest and outcome</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mi>Y</mi></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub></mrow></math> </ephtml> , and <emph>(PRETEST)<subs>ij</subs></emph> is each student's standardized pretest measure. In this formulation, the model intercept,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>α</mi></mrow></math> </ephtml> , represents the grand mean for the control group members with average pretest scores (reference group),</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mi>u</mi></mrow><mrow><mi>j</mi></mrow></msub></mrow><mo /></math> </ephtml> represents the school-specific error, and</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mi>ε</mi></mrow><mrow><mtext mathvariant="italic">ij</mtext><mo /></mrow></msub></mrow></math> </ephtml> represents the student-specific error.</p> <hd id="AN0187023105-24">Internal Validity of Impact Estimates</hd> <p>We consider two key factors that bear on the internal validity of our impact estimates: sample attrition and baseline equivalence of the analytical sample. With regard to attrition, of the 166 sampled students, only 11 students (6.63%) were lost from the analysis of <emph>IdO</emph> outcomes, and eight students (4.22%) had incomplete assessment data and were excluded from the analysis of the <emph>Logramos</emph> outcomes. Differential attrition across the treatment and control conditions was minimal, with two of 84 treatment students and seven of 82 control students having missing outcome data for the <emph>IdO</emph> assessment. Similarly, one of the 84 treatment students and four of the 82 control students had missing data for the <emph>Logramos</emph> test. In terms of overall attrition rates for <emph>IdO</emph> and <emph>Logramos</emph> and differential attrition rates for treatment and control students, both met WWC standards for "low" attrition and a minimal risk of bias (WWC, [<reflink idref="bib66" id="ref53">66</reflink>]).</p> <p>Finally, regarding baseline equivalence of the treatment and control samples, the pretest score on the <emph>IdO</emph> Language Total outcome was 464.34 (<emph>SD</emph> = 34.60) for treatment students and 454.95 (<emph>SD</emph> = 41.76) for control students, resulting in a standardized mean difference of 0.245. The results for the <emph>Logramos</emph> Language Arts Total outcome revealed a mean score of 143.83 (<emph>SD</emph> = 52.19) for treatment students and 139.27 (<emph>SD</emph> = 57.38) for control students, which was equivalent to a standardized mean difference of 0.08. In both cases, these results for baseline equivalence met the WWC ([<reflink idref="bib66" id="ref54">66</reflink>]) standards for baseline equivalence.</p> <hd id="AN0187023105-25">Calculating Cost-Effectiveness Ratios</hd> <p>To calculate CERs for the DLL program, we compared the impact estimates with our estimate for the average annual per student cost. Specifically, we divided the estimated impact of the DLL intervention for the cohort of students (in all participating districts) who participated in the program in 2017–18 by the average annual per-student DLL implementation cost. We considered two outcome measures in our analysis: (a) <emph>Logramos'</emph> overall total Language Arts score and (b) the total score for the <emph>IdO</emph> assessment.</p> <p>We calculated CERs in two ways. We first calculated the CER as the impact of DLL that may be achieved for an investment of $1,000:</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mtext mathvariant="italic">CER</mtext><mi mathvariant="italic">=</mi><mrow><mfrac><mrow><mtext mathvariant="italic">DLL Effect</mtext><mo /><mtext mathvariant="italic">Size</mtext></mrow><mrow><mtext mathvariant="italic">Average Per</mtext><mo>−</mo><mtext mathvariant="italic">Student Implementation</mtext><mo /><mtext mathvariant="italic">Cost</mtext></mrow></mfrac></mrow><mo /><mo>*</mo><mo /><mn>1000</mn></mrow></math> </ephtml> </p> <p>This standardized CER, which provides the estimated impact of DLL associated with a $1,000 investment in the program, allows practitioners to easily compare programs aimed at the same learning goals, even if the programs are otherwise different from one another in terms of costs, impacts, or features of program startup and implementation.</p> <p>As an alternate calculation, we estimated the cost per 1 <emph>SD</emph> impact of DLL:</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mtext mathvariant="italic">CER</mtext><mo>=</mo><mrow><mfrac><mn>1</mn><mrow><mtext mathvariant="italic">DLL Effect</mtext><mo /><mtext mathvariant="italic">Size</mtext></mrow></mfrac></mrow><mtext>*</mtext><mtext mathvariant="italic">Per</mtext><mo>−</mo><mtext mathvariant="italic">Student Implementation</mtext><mo /><mtext mathvariant="italic">Cost</mtext></mrow></math> </ephtml> </p> <p>Like an effect size, both estimates place all cost-effectiveness information in the same metric, which can facilitate comparisons across interventions. However, some caution in interpretation is warranted. The former metric does not necessarily assume incremental impacts associated with each additional $1,000 investment in the intervention, and the latter estimate does not assume that obtaining the 1 <emph>SD</emph> impact would necessarily be produced by the incremental investment.</p> <hd id="AN0187023105-26">Results</hd> <p>Table 3 presents findings that describe the cost of implementing the DLL program in the participating districts, overall and for specific resource types. The average cost per student for the intervention was $7,120, of which $713 represented onetime startup costs, and $6,407 was for the recurring costs of operating the DLL program. Personnel costs represented 93% of the average cost per student, with time spent by DLL teachers comprising 79% of the program's costs and DLL teacher leaders' time comprising about 14% of the cost of implementing the DLL program. On average, districts interested in establishing a DLL program might expect to incur an initial onetime cost of approximately $250 per student in tuition, fees, and transportation for DLL teacher leaders and teachers to initiate the program and then incur another $48 per student, per year, in ongoing costs for these items. Similarly, districts might expect to incur about $86 per student in startup costs for children's books and other materials, DLL teacher leader textbooks, and DLL teacher textbooks and an annual recurring cost of $23 per student for these items.</p> <p>Table 3. DLL program costs, overall and by ingredient (in 2022 Dollars).</p> <p> <ephtml> <table><thead><tr><td>Ingredients</td><td>Total annual cost</td><td>Average annual cost per student<xref ref-type="table-fn" rid="tfn5">a</xref></td><td>Total costs (%)</td></tr><tr><td>Startup</td><td>Recurring</td><td>Overall</td><td>Startup</td><td>Recurring</td><td>Overall</td></tr></thead><tbody valign="top"><tr><td>Total cost</td><td char=".">141,876</td><td char=".">1,274,982</td><td char=".">1,416,858</td><td char=".">713</td><td char=".">6,407</td><td char=".">7,120</td><td char=".">100.0%</td></tr><tr><td><bold>Personnel (total)</bold></td><td char="."><bold>72,845</bold></td><td char="."><bold>1,245,231</bold></td><td char="."><bold>1,318,076</bold></td><td char="."><bold>366</bold></td><td char="."><bold>6,257</bold></td><td char="."><bold>6,623</bold></td><td char="."><bold>93.0%</bold></td></tr><tr><td><italic>Staff time to implement DLL program in districts</italic></td><td /><td /><td /><td /><td /><td /><td /></tr><tr><td>Principal</td><td char=".">4,842</td><td char=".">4,842</td><td char=".">9,683</td><td char=".">24</td><td char=".">24</td><td char=".">49</td><td /></tr><tr><td>Classroom teacher</td><td char=".">440</td><td char=".">440</td><td char=".">879</td><td char=".">2</td><td char=".">2</td><td char=".">4</td><td /></tr><tr><td>Education specialist</td><td char=".">745</td><td char=".">745</td><td char=".">1,490</td><td char=".">4</td><td char=".">4</td><td char=".">7</td><td /></tr><tr><td>DLL teacher duties</td><td>—</td><td char=".">962,917</td><td char=".">962,917</td><td>—</td><td char=".">4,839</td><td char=".">4,839</td><td /></tr><tr><td>DLL teacher leader duties</td><td /><td char=".">129,101</td><td char=".">129,101</td><td>—</td><td char=".">649</td><td char=".">649</td><td /></tr><tr><td><italic>Initial conference and training</italic></td><td /><td /><td /><td>—</td><td>—</td><td /><td /></tr><tr><td>DLL teacher leader initial conference</td><td char=".">1,124</td><td /><td char=".">1,124</td><td char=".">6</td><td>—</td><td char=".">6</td><td /></tr><tr><td>DLL teacher initial conference</td><td char=".">12,136</td><td /><td char=".">12,136</td><td char=".">61</td><td>—</td><td char=".">61</td><td /></tr><tr><td>DLL teacher leader training time</td><td char=".">10,407</td><td /><td char=".">10,407</td><td char=".">52</td><td>—</td><td char=".">52</td><td /></tr><tr><td>DLL teacher training time</td><td char=".">43,152</td><td /><td char=".">43,152</td><td char=".">217</td><td>—</td><td char=".">217</td><td /></tr><tr><td><italic>Ongoing professional development (PD)</italic></td><td /><td /><td /><td>—</td><td>—</td><td /><td /></tr><tr><td>DLL teacher leader ongoing PD</td><td /><td char=".">35,025</td><td char=".">35,025</td><td>—</td><td char=".">176</td><td char=".">176</td><td /></tr><tr><td>DLL teacher ongoing PD</td><td /><td char=".">46,152</td><td char=".">46,152</td><td>—</td><td char=".">232</td><td char=".">232</td><td /></tr><tr><td><italic>Ongoing conferences</italic></td><td /><td /><td /><td>—</td><td>—</td><td /><td /></tr><tr><td>DLL teacher leader conferences</td><td /><td char=".">15,649</td><td char=".">15,649</td><td>—</td><td char=".">79</td><td char=".">79</td><td /></tr><tr><td>DLL teacher conferences</td><td /><td char=".">50,360</td><td char=".">50,360</td><td>—</td><td char=".">253</td><td char=".">253</td><td /></tr><tr><td><bold>Facilities (total)</bold></td><td char="."><bold>2,248</bold></td><td char="."><bold>15,429</bold></td><td char="."><bold>17,676</bold></td><td><bold>11</bold></td><td><bold> 77</bold></td><td char="."><bold>89</bold></td><td char="."><bold>1.2%</bold></td></tr><tr><td>One-way glass room</td><td char=".">1,046</td><td /><td char=".">1,046</td><td char=".">5</td><td>—</td><td char=".">5</td><td /></tr><tr><td>Room for DLL initial training</td><td char=".">1,201</td><td /><td char=".">1,201</td><td char=".">6</td><td>—</td><td char=".">6</td><td /></tr><tr><td>Room for DLL ongoing PD</td><td>—</td><td char=".">52</td><td char=".">52</td><td>—</td><td char=".">0</td><td char=".">0</td><td /></tr><tr><td>Space for DLL lessons</td><td /><td char=".">9,610</td><td char=".">9,610</td><td>—</td><td char=".">48</td><td char=".">48</td><td /></tr><tr><td>DLL teacher leader office space at district</td><td /><td char=".">5,766</td><td char=".">5,766</td><td>—</td><td char=".">29</td><td char=".">29</td><td /></tr><tr><td><bold>Supplies and equipment (total)</bold></td><td char="."><bold>17,040</bold></td><td char="."><bold>4,739</bold></td><td char="."><bold>21,779</bold></td><td><bold>86</bold></td><td><bold>23</bold></td><td char="."><bold>109</bold></td><td char="."><bold>1.5%</bold></td></tr><tr><td>Children's books and other materials</td><td char=".">14,166</td><td /><td char=".">14,166</td><td char=".">71</td><td>—</td><td char=".">71</td><td /></tr><tr><td>DLL teacher leader textbooks</td><td char=".">1,125</td><td /><td char=".">1,125</td><td char=".">6</td><td>—</td><td char=".">6</td><td /></tr><tr><td>DLL teacher textbooks</td><td char=".">1,749</td><td /><td char=".">1,749</td><td char=".">9</td><td>—</td><td char=".">9</td><td /></tr><tr><td>DL teacher leader office computer</td><td /><td char=".">661</td><td char=".">661</td><td>—</td><td char=".">3</td><td /><td /></tr><tr><td>DLL teacher school computer</td><td /><td char=".">4,078</td><td char=".">4,078</td><td>—</td><td char=".">20</td><td char=".">20</td><td /></tr><tr><td><bold>Other inputs (total)</bold></td><td char="."><bold>49,743</bold></td><td char="."><bold>9,584</bold></td><td char="."><bold>59,327</bold></td><td char="."><bold>257</bold></td><td><bold> 41</bold></td><td char="."><bold>298</bold></td><td char="."><bold>4.2%</bold></td></tr><tr><td>Tuition</td><td char=".">7,438</td><td char=".">1,064</td><td char=".">8,502</td><td char=".">37</td><td char=".">5</td><td char=".">43</td><td /></tr><tr><td>Fees</td><td char=".">42,305</td><td char=".">5,965</td><td char=".">48,271</td><td char=".">213</td><td char=".">30</td><td char=".">243</td><td /></tr><tr><td>Transportation</td><td>—</td><td char=".">2,554</td><td char=".">2,554</td><td char=".">7</td><td char=".">6</td><td char=".">13</td><td /></tr></tbody></table> </ephtml> </p> <ulist> <item>4 <emph>Note.</emph> DLL: Descubriendo la Lectura. In some instances, the totals across rows and columns may not equal the reported sums in the table because of rounding in whole dollar amounts.</item> <item>5 The average cost per student was calculated as the total cost (overall, startup, and ongoing) divided by the total number of students who received the DLL intervention during the 2017–18 school year at selected sites (<emph>n</emph> = 199), reported by districts to the International Data Evaluation Center.</item> </ulist> <p>The average annual cost per student varied considerably across sites, ranging from $4,504 to $11,856 per student (Figure 1). In large part, the differences in costs across sites are explained by the number of students at a given site; sites with fewer students receiving DLL had higher per-student costs, and sites with larger numbers of participating students had lower per-student costs. This deviation is because some DLL expenses (such as initial trainings) are fixed, regardless of whether one or eight students participate in DLL. The other cost driver was the amount of time that DLL teachers spent working with students. For instance, although Site C had the second largest number of students participating in the DLL program, its annual per-student cost was substantially higher than other sites because of the comparatively large number of hours that DLL teachers spent working with students. It also was the case that some sites did not have a DLL teacher leader for the study year, so these sites had lower overall personnel costs.</p> <p>Graph: Figure 1. Average per-student cost, by district site (2022 Dollars). Note. Sites A to G represent the seven districts that participated in the cost study. District sites are presented from left to right in order from the site with the smallest to the largest number of students served. The national average represents the overall cost across the seven districts divided by the total number of students (n = 199).</p> <hd id="AN0187023105-27">Cost-Effectiveness Ratio</hd> <p>To estimate the DLL program's cost-effectiveness, we combined the estimated achievement impacts and per-pupil costs. The estimated impact of DLL on the <emph>IdO</emph> total score was <emph>d</emph> = 0.91, with a standard error of 0.15; the total Language Arts score for the <emph>Logramos</emph> was <emph>d</emph> = 0.23, with a standard error of 0.09 (Table 4). Both estimated impacts were statistically significant. To calculate CERs, we multiplied the impact estimates by $1,000 and divided by the overall average annual per-student cost ($7,120). The resulting CERs suggest that effect sizes of <emph>d</emph> = 0.13 for the <emph>IdO</emph> and <emph>d</emph> = 0.03 for the <emph>Logramos</emph> are associated with a $1,000 investment in the program (Table 4).</p> <p>Table 4. DLL impact estimates and cost-effectiveness ratios.</p> <p> <ephtml> <table><thead><tr><td>Assessment</td><td>Effect size</td><td>Effect per $1,000</td><td>Cost for 1 <italic>SD</italic> impact</td></tr><tr><td /><td>Average cost per student</td><td>District with highest per-student cost</td><td>District with lowest per-student cost</td><td>Average cost per student</td><td>District with highest per-student cost</td><td>District with lowest per-student cost</td></tr></thead><tbody valign="top"><tr><td><italic>Instrumento de Observación</italic></td><td char=".">0.91*** (0.15)</td><td char=".">0.13</td><td char=".">0.08</td><td char=".">0.20</td><td>$7,824</td><td>$13,028</td><td>$4,949</td></tr><tr><td><italic>Logramos</italic></td><td char=".">0.23<xref ref-type="table-fn" rid="tfn7">*</xref> (0.09)</td><td char=".">0.03</td><td char=".">0.02</td><td char=".">0.05</td><td>$30,957</td><td>$51,547</td><td>$19,582</td></tr></tbody></table> </ephtml> </p> <ulist> <item>6 <emph>Note.</emph> DLL: Descubriendo la Lectura. The average cost per student for all districts is $7,120; for the district with the highest per-student cost, $11,856; for the district with the lowest per-student cost, $4,504.</item> <item>7 <emph>p</emph> <.05, ***<emph>p</emph> <.001.</item> </ulist> <p>For the second CER estimate, we calculated the cost associated with raising achievement by 1 <emph>SD</emph>. A 1 <emph>SD</emph> increase in student achievement, as measured by the <emph>IdO</emph>, is about $7,824 per student, whereas a 1 <emph>SD</emph> increase in student achievement using the <emph>Logramos</emph> achievement measure is $30,957 per student.</p> <p>We checked the sensitivity of the CERs to alternate assumptions about the cost of implementing the DLL program using the per-student cost for the districts with the highest ($11,856) and lowest ($4,504) cost estimates. Because the variation in cost among sites was largely caused by differences in economies of scale, these additional analyses illustrate how the DLL program's CERs vary according to the number of students in a district who receive the DLL intervention. We found effect sizes for the <emph>IdO</emph> falling between <emph>d</emph> = 0.08 and <emph>d =</emph> 0.20 for a $1,000 investment in the program and between <emph>d =</emph> 0.02 and <emph>d =</emph> 0.05 when achievement is measured using the <emph>Logramos</emph> measure. Similarly, we found large differences in the cost for a 1 <emph>SD</emph> increase in student achievement: between $4,949 and $13,028 for achievement gains on the <emph>IdO</emph> and between $19,582 and $51,547 for gains on the <emph>Logramos.</emph> Taken together, these findings suggest that districts offering the DLL intervention to a larger number of students benefit from additional efficiencies in program implementation than districts with smaller DLL programs.</p> <hd id="AN0187023105-28">Discussion</hd> <p>Relatively recently, researchers and policymakers have benefited from more nuanced and realistic estimates of the typical impacts of educational programs, policies, and interventions. Moving beyond the general benchmarks noted by Cohen ([<reflink idref="bib17" id="ref55">17</reflink>]), which characterize effect sizes of 0.20, 0.50, and 0.80 as, respectively, "small," "medium," and "large," educational researchers have noted important caveats when assessing the magnitudes of the effects of education interventions (Baird & Pane, [<reflink idref="bib3" id="ref56">3</reflink>]; Hill et al., [<reflink idref="bib27" id="ref57">27</reflink>]; Kraft, [<reflink idref="bib34" id="ref58">34</reflink>]; Lipsey et al., [<reflink idref="bib39" id="ref59">39</reflink>]). These advances offer educational researchers and policymakers more informative guidance for evaluating and interpreting intervention effects. Similar benchmarks and interpretations of interventions' costs and impacts have been far more elusive. Although there have been a few efforts to analyze and compare the relative cost-effectiveness or cost-benefit ratios of, for instance, social-emotional learning programs (Belfield et al., [<reflink idref="bib6" id="ref60">6</reflink>]), early literacy programs (Borman & Hewes, [<reflink idref="bib7" id="ref61">7</reflink>]; Hollands et al., [<reflink idref="bib30" id="ref62">30</reflink>]), and other education policies and interventions (Harris, [<reflink idref="bib25" id="ref63">25</reflink>]), there is little specific guidance to interpret the magnitudes of the CERs reported here.</p> <p>Beyond the descriptions of the costs, impacts, and CERs presented here, how can we assess their practical significance in context? One starting point to contextualize the costs and impacts of DLL can be gleaned from the recent article by Kraft ([<reflink idref="bib34" id="ref64">34</reflink>]), which suggests that a high per-pupil cost is $4,000 or more and a large effect size is 0.20 or greater. Thus, DLL might be understood as an intervention that has both substantial impacts and substantial costs. More specifically, by examining the distribution of impacts across 242 recent RCTs of education interventions and a range of cost estimates provided by Kraft ([<reflink idref="bib34" id="ref65">34</reflink>]), the average impacts of DLL on the <emph>Logramos</emph> and <emph>IdO</emph> assessments are, respectively, at the 76th percentile and beyond the 99th percentile, whereas the costs, in 2022 dollars, are between the 70th and 80th percentiles of the overall distribution.</p> <p>To provide further context, one may consider DLL CERs relative to those calculated by Yeh ([<reflink idref="bib68" id="ref66">68</reflink>]) for 22 educational programs and policies. Yeh provided a list of CERs for a variety of alternate resource allocations, including expenditures devoted to computer-assisted instruction, higher teacher salaries, summer school programs, class-size reductions, full-day kindergarten, and various preschool programs. The median CER for the reading outcomes presented by Yeh was class-size reduction, with an impact per $1,000 of <emph>d</emph> = 0.06, when converted to constant 2022 dollars.</p> <p>Finally, two recent papers investigated the cost-effectiveness of interventions that, as with DLL, target readers who are struggling in the early elementary grades. For instance, Shrestha et al. ([<reflink idref="bib52" id="ref67">52</reflink>]) reported the costs of RR, along with costs for the 18 other most frequently implemented interventions offered to students in lieu of RR. Similar to the estimates reported here, they found RR's cost per student ranged from approximately $5,500 to $10,300. Relative per-pupil costs for the other interventions varied considerably. Teacher-led interventions ranged from approximately $700 to nearly $15,000 per student, depending on several factors, the most important one being group size, wherein one-to-one (i.e., teacher-to-student) interventions (e.g., DLL, RR) had the highest costs per student. Technology-based interventions delivered directly to students on a tablet or computer had the lowest costs per student, with many costing approximately $200 per student and some costing less than $100 per student.</p> <p>Combining cost and impact data, Hollands et al. ([<reflink idref="bib30" id="ref68">30</reflink>]) investigated the relative cost-effectiveness of seven early literacy interventions: Kindergarten Peer-Assisted Learning Strategies (K-PALS), Stepping Stones to Literacy, Sound Partners, Fast ForWord, RR, Corrective Reading, and the Wilson Reading System. For outcomes from the alphabetics literacy domain, the incremental CERs to obtain a unit increase in effect size ranged from a low of $38 for K-PALS to a high of $38,135 for Corrective Reading. A second literacy domain, fluency, revealed CERs to obtain a unit increase in effect size ranged from a low of $165 for Sound Partners to a high of $6,364 for Corrective Reading. Although these various comparisons provide useful preliminary information, more specific cost-effectiveness comparisons for other interventions that target Grade 1 and Spanish-language literacy outcomes can permit a closer "apples-to-apples" comparison.</p> <hd id="AN0187023105-29">Comparing DLL Costs and Effects With Other Programs Targeting Spanish-Language Literacy Outco...</hd> <p>Unfortunately, a dearth of impact evaluations, let alone cost-effectiveness analyses, exist for early literacy interventions designed to improve Grade 1 students' Spanish-language outcomes. Our scan of the WWC website,[<reflink idref="bib9" id="ref69">9</reflink>] multiple meta-analyses (Ludwig et al., [<reflink idref="bib40" id="ref70">40</reflink>]; Richards-Tutor et al., [<reflink idref="bib49" id="ref71">49</reflink>]), and research reviews (August et al., [<reflink idref="bib1" id="ref72">1</reflink>]; August & Shanahan, [<reflink idref="bib2" id="ref73">2</reflink>]; Baker et al., [<reflink idref="bib4" id="ref74">4</reflink>]; Cheung & Slavin, [<reflink idref="bib13" id="ref75">13</reflink>], [<reflink idref="bib14" id="ref76">14</reflink>]; Gersten et al., [<reflink idref="bib23" id="ref77">23</reflink>]; Snyder et al., [<reflink idref="bib57" id="ref78">57</reflink>]) revealed only two early elementary programs with evidence of impact on the Spanish-language literacy outcomes of struggling readers. One of the most recent reviews concluded that there were "dramatically" fewer studies of high-quality reading interventions for ELs compared to those for native-English speakers (Richards-Tutor et al., [<reflink idref="bib49" id="ref79">49</reflink>], p. 161).</p> <p> <emph>Enhanced Proactive Reading</emph> (currently distributed by McGraw-Hill Education as <emph>Intervenciones tempranas de la lectura</emph>) is one example of a reading program with some evidence of efficacy promoting Spanish-language outcomes for bilingual students (What Works Clearinghouse, [<reflink idref="bib65" id="ref80">65</reflink>]). Similar to DLL, the program is designed to meet the needs of first-grade ELs experiencing problems with learning to read through conventional instruction. Rather than a one-to-one tutoring program, the comprehensive literacy curriculum is implemented as small-group daily reading instruction. Vaughn, Linan-Thompson, et al. ([<reflink idref="bib64" id="ref81">64</reflink>]) and Vaughn, Cirino, et al. ([<reflink idref="bib63" id="ref82">63</reflink>]) evaluated the program in three sites in Texas, with the latter study (89 students) replicating the former (69 students). Small sample sizes limited the evidence gained, but both studies revealed impacts on several reading measures, with an average effect size in the replication study of 0.43. These studies, however, are 15 years old, and there is no systematic evidence available concerning the costs to support new implementations. Thus, although the <emph>Enhanced Proactive Reading</emph> model provides a comparative impact estimate, its cost-effectiveness cannot be determined.</p> <p>Additional evidence comes from two studies of Spanish-language adaptations of the widely implemented Success for All program in California and Houston bilingual schools. The Spanish-language version of Success for All, <emph>Exito Para Todos</emph>, is a comprehensive reform program for elementary schools. Similar to DLL, it focuses on developing the early literacy skills of ELs. The program includes a tutoring component, a well-specified reading curriculum along with regular formative reading assessments, a parent outreach component, ongoing teacher professional development, and instruction in English as a second language. As a whole-school approach, Success for All/<emph>Exito Para Todos</emph> restructures supplementary federal and state staff and resources to focus on prevention, early intervention, and long-term professional development, rather than remediation. Dianda and Flaherty ([<reflink idref="bib19" id="ref83">19</reflink>]) reported the results of the California-based implementation, which included 25 Success for All/<emph>Exito Para Todos</emph> ELs and 41 matched comparison students attending schools with bilingual programs. Although Cheung and Slavin ([<reflink idref="bib13" id="ref84">13</reflink>]) reported a statistically significant pretest difference equivalent to an effect size of <emph>d</emph> = 0.19 favoring the Success for All/<emph>Exito Para Todos</emph> students, the intervention impacts of <emph>d</emph> = 1.03 were substantial. The Houston study, conducted by Nunnery et al. ([<reflink idref="bib47" id="ref85">47</reflink>]), included 201 Success for All/<emph>Exito Para Todos</emph> students and 102 comparison students. The reported quasi-experimental impact was equivalent to an effect size of <emph>d</emph> = 0.22. Combining these two impact estimates as weighted effect sizes, with weights defined as</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mi>w</mi><mo>=</mo><mrow><mfrac><mrow><mn>1</mn></mrow><mrow><mrow><msup><mrow><mtext mathvariant="italic">SE</mtext></mrow><mrow><mn>2</mn></mrow></msup></mrow></mrow></mfrac></mrow></math> </ephtml> , the average weighted impact of Success for All/<emph>Exito Para Todos</emph> on Spanish-language outcomes is <emph>d</emph> = 0.36.[<reflink idref="bib10" id="ref86">10</reflink>]</p> <p>Unlike <emph>Enhanced Proactive Reading</emph>, there have been several efforts to estimate costs for implementing Success for All. Even though no estimates specifically exist for the Spanish-language version of the program, the components of the Spanish- and English-language programs are identical, and the resources demanded by the two programs should remain comparable. The first estimate comes from Borman and Hewes ([<reflink idref="bib7" id="ref87">7</reflink>]), who evaluated the long-term impacts and initial program costs based on data from the five original Baltimore, Maryland, Success for All schools. The effect per $1,000 investment in Success for All, in 2000 dollars, reported by Borman and Hewes was <emph>d</emph> = 0.12, which is equivalent to a CER of <emph>d</emph> = 0.07 when converted to constant 2022 dollars using gross domestic product implicit price deflators. However, this CER is based on costs accumulated over several years of implementation and sustained impacts for students through Grade 7. Thus, it is not directly comparable to the DLL CER estimates. Further, these estimates are specific to the five original Baltimore schools, which devoted differing amounts of resources and personnel to Success for All, ranging from $195,293 to $620,548, with an average of $353,820 in 2000 dollars or $600,582 in constant 2022 dollars. They do, however, represent real-world differences in the differing ways that schools elect to implement Success for All. Because the schoolwide Success for All program targeted all students in Grades K–5, Borman and Hewes used the school-by-school student enrollments in these grades to calculate the annual per-pupil cost of $795, or $1,349 in constant 2022 dollars.</p> <p>Another cost estimate for Success for All was generated by King ([<reflink idref="bib33" id="ref88">33</reflink>]). This estimate considered an ideal full implementation of the program, and the costs, as expected, are somewhat higher than those provided by Borman and Hewes ([<reflink idref="bib7" id="ref89">7</reflink>]). In addition, King's estimates include costs associated with parent volunteers, which was not a personnel resource used in the five original Baltimore schools in the Borman and Hewes analysis. Harris ([<reflink idref="bib25" id="ref90">25</reflink>]) recalculated King's original estimates, providing some additional details and costs associated with parent and staff time. After adjusting King's original estimates, Harris concluded that the average annual per-pupil cost for an elementary school with a student enrollment of 500 ranges from $1,182 to $2,318, or $1,728 to $4,428 in 2022 dollars. Using impact estimates from a randomized study conducted by Borman et al. ([<reflink idref="bib8" id="ref91">8</reflink>]), which yielded effect sizes ranging from <emph>d</emph> = 0.20 to <emph>d</emph> = 0.33, Harris concluded that the effect size per $1,000 investment in Success for All, expressed in constant 2022 dollars, was <emph>d</emph> = 0.05 to <emph>d</emph> = 0.19.</p> <p>Finally, using average impact estimates gleaned from the WWC and cost estimates generated from a review of several implementations of Success for All, Simon ([<reflink idref="bib53" id="ref92">53</reflink>]) calculated cost-effectiveness estimates in her dissertation exploring the relative costs and effects of several early reading programs. Applying varying discount rates and assumptions, Simon concluded that the effect per $1,000 investment in Success for All, in constant 2007 dollars, was between <emph>d</emph> = 0.021 and 0.024. Converting Simon's estimated per-pupil cost of $11,706, which employed a 3.5% discount rate, the CER in constant 2022 dollars of $17,113 is <emph>d</emph> = 0.02.</p> <p>A significant reason that the Simon ([<reflink idref="bib53" id="ref93">53</reflink>]) per-pupil cost estimates are considerably larger than those from both Borman and Hewes ([<reflink idref="bib7" id="ref94">7</reflink>]) and Harris ([<reflink idref="bib25" id="ref95">25</reflink>]) is because Simon considered only students enrolled in Grades K–2 schools, which is roughly half the schoolwide enrollment of a typical K–5 elementary school. Simon noted that the effectiveness data came from Grades K–2 only and, therefore, correspondingly focused the total schoolwide K–5 costs on only the K–2 students. Although Simon is correct that the impact data used did not include Grades 3–5, the estimated schoolwide costs did. If the estimated costs are considered across the full K–5 student enrollment, then the per-pupil annual cost is approximately $8,556 in constant 2022 dollars.[<reflink idref="bib11" id="ref96">11</reflink>]</p> <p>Taken together, the constant 2022 per-pupil cost estimates for prior implementations of Success for All range from $1,394, as reported by Borman and Hewes ([<reflink idref="bib7" id="ref97">7</reflink>]), to $17,113 reported by Simon ([<reflink idref="bib53" id="ref98">53</reflink>]). The average quasi-experimental impact estimate from the two prior studies of <emph>Exito Para Todos</emph> is <emph>d</emph> = 0.36. In Table 5, we summarize the per-student costs, impacts, and CERs for DLL and Success for All/<emph>Exito Para Todos</emph>. The DLL CERs are clearly within the same estimated range as those for Success for All/<emph>Exito Para Todos</emph>, with the two DLL CERs being more favorable than the CER based on the highest estimated per-pupil cost of Success for All provided by Simon, but less favorable than the two lowest per-pupil cost estimates from Borman and Hewes and from Harris ([<reflink idref="bib25" id="ref99">25</reflink>]).</p> <p>Table 5. DLL and success for all/exito para todos cost-effectiveness ratios for spanish-language literacy outcomes summarized in constant 2022 dollars.</p> <p> <ephtml> <table><thead><tr><td>Intervention</td><td>Source of cost estimate</td><td>Per-student cost</td><td>Effect size</td><td>Effect per $1,000</td><td>Cost for 1 <italic>SD</italic> impact</td></tr></thead><tbody valign="top"><tr><td>DLL</td><td /><td /><td /><td /><td /></tr><tr><td /><td>Current study (<italic>Instrumento de Observación</italic>)</td><td>$7,120</td><td char=".">0.91</td><td char=".">0.13</td><td>$7,824</td></tr><tr><td /><td>Current Study (<italic>Logramos</italic>)</td><td>$7,120</td><td char=".">0.23</td><td char=".">0.03</td><td>$30,957</td></tr><tr><td>Success for All/ <italic>Exito Para Todos</italic></td><td>Borman & Hewes (<xref ref-type="bibr" rid="bibr7">2002</xref>)</td><td>$1,394</td><td char=".">0.36</td><td char=".">0.26</td><td>$3,872</td></tr><tr><td>Harris (<xref ref-type="bibr" rid="bibr25">2009</xref>)</td><td>$1,728</td><td char=".">0.36</td><td char=".">0.21</td><td>$4,800</td></tr><tr><td>Harris (<xref ref-type="bibr" rid="bibr25">2009</xref>)</td><td>$4,428</td><td char=".">0.36</td><td char=".">0.08</td><td>$12,300</td></tr><tr><td>Simon (<xref ref-type="bibr" rid="bibr53">2011</xref>)</td><td>$8,556</td><td char=".">0.36</td><td char=".">0.04</td><td>$23,767</td></tr><tr><td>Simon (<xref ref-type="bibr" rid="bibr53">2011</xref>)</td><td>$17,113</td><td char=".">0.36</td><td char=".">0.02</td><td>$47,536</td></tr></tbody></table> </ephtml> </p> <p>8 <emph>Note.</emph> DLL: Descubriendo la Lectura. The per-student cost of $17,113 from Simon ([<reflink idref="bib53" id="ref100">53</reflink>]) is derived by dividing the total cost of Success for All/<emph>Exito Para Todos</emph> by the school's K–2 enrollment, and the estimate of $8,556 is derived by dividing the total cost by the school's full K–5 enrollment.</p> <hd id="AN0187023105-30">Limitations</hd> <p>The fields of education and economics have generated little evidence concerning impacts of Spanish-language literacy programs for struggling readers. Few interventions of this type have evidence of impacts and even fewer have rigorous estimates of costs. We attempted to address both shortcomings with this study. Nevertheless, we have few comparisons of costs and impacts from other well-specified strategies or programs to advance Spanish-language literacy. As a result, a great deal more research, documenting both costs and impacts, is necessary to place our results in context. Is a $7,120 investment that, on average, advances the achievement of students by nearly <emph>d</emph> of 0.91 a sound investment? Given this achievement impact, on average, a DLL student is expected to outperform 82% of similar control students not receiving DLL. On the face of it, such an investment with a greater than 80% success rate seems quite worthy. Nevertheless, far more examples are needed.</p> <p>Two limitations regarding DLL's potential opportunity costs for students and how the program operates within the larger context of the school are worth discussing. First, when students participate in DLL during the school day and miss nonliteracy instruction within their regular classrooms, they may miss some valuable instruction in other domains, including, for instance, mathematics, science, physical education, art, or other social-emotional learning opportunities. Without measures of students' outcomes within these other domains, we cannot be sure whether students missed some other valuable opportunities to learn and develop. Learning to be a good reader, though, enables one to gain access to and learn about content across all academic disciplines and is a foundational skill. For instance, Borman and Hewes ([<reflink idref="bib7" id="ref101">7</reflink>]) provided evidence suggesting that the Success for All intervention not only improved students' reading achievement but also had modest yet statistically significant impacts on students' mathematics achievement despite the increased time and focus on literacy instruction and potential opportunity costs associated with a reduced focus on mathematics instruction.</p> <p>Second, our study is limited in terms of the extent to which we gathered detailed information about the counterfactual condition. All control students in our sample were actually wait-listed students served by DLL in the spring, after the fall treatment students received program services. This is a typical practice in all DLL schools—a first group of students is served during the fall, and a second group is served after the first group has completed the intervention. As a result, the standard practice is for the wait-listed students to receive only regular classroom literacy instruction in the fall and DLL in the spring. With this arrangement, the wait-listed students do not typically receive alternate supplemental literacy treatments, which also may have associated costs. Anecdotally, in working with the DLL teachers and teacher leaders, we understand that few students receive additional services when the first group of fall participants (our treatment students) receive DLL services. However, there may be some instances when this standard practice is not followed. To the extent that we missed potential participation of our wait-list control students in other supplemental literacy interventions, we have overestimated the incremental costs of DLL and potentially underestimated its impacts relative to an assumed no-treatment-counterfactual condition.</p> <p>Few education studies have been conducted combining cost and impact data from the same samples of students, teachers, and schools (Belfield & Bowden, [<reflink idref="bib5" id="ref102">5</reflink>]). Indeed, the vast majority of studies that investigate intervention costs along with their impact estimates, including several prominent articles cited in this paper (e.g., Belfield et al., [<reflink idref="bib6" id="ref103">6</reflink>]; Hollands et al., [<reflink idref="bib30" id="ref104">30</reflink>]; King, [<reflink idref="bib33" id="ref105">33</reflink>]) rely on prior estimates of intervention impacts combined with the authors' independent estimates of costs. Our study is one of the few to combine the same samples from which researchers collected both impact and cost data simultaneously. Nevertheless, the impact and cost samples reported here do not overlap entirely. The cost data come from 199 students across seven districts, and the impact results are from 166 students in 10 districts.</p> <p>Our intention is to provide the most comprehensive and inclusive estimates of both costs and impacts, but if we instead limited the impact estimates to the 146 students with complete outcome data from the seven districts included in the cost study, the results would remain essentially the same: <emph>d</emph> = 0.90 for the <emph>IDO</emph> outcome and <emph>d</emph> = 0.21 for the <emph>Logramos</emph> assessment. Also, even though we do not have access to personnel costs from three schools and districts that served 20 students in our impact sample, we do know that the overall average number of DLL sessions received by the 166 students from the impact sample and 199 students from the cost sample also were essentially equal at, respectively, 52 and 51 sessions. Similarly, the 146 students with complete impact data from the seven districts involved in the cost study received 52 DLL sessions, whereas the 199 students in the cost study from the same districts participated in 51 sessions. The number of sessions is a key student-level variable that affects DLL costs, and, of course, the estimated impacts directly affect our CERs. On the other hand, though, the three districts that were not included in our cost sample had relatively small numbers of students from whom we collected achievement data. As discussed, and as shown in Figure 1, larger district sites tended to achieve economies of scale. We concede this lack of overlap between the cost and impact studies. Though the relative achievement impacts for the full sample and the smaller cost-study sample are equivalent, it is possible that costs were higher in the three districts that did not contribute to the cost study.</p> <p>Finally, our study spans only 1 year and includes only a sample from the population of DLL schools. During the 2017–18 school year, 470 students across eight states were served by DLL.[<reflink idref="bib12" id="ref106">12</reflink>] Our sample did not include schools from three states: Colorado, Minnesota, and Washington. Our sample did include more than one third of the population of DLL students and schools and more than half of the states served by the program. This rather substantial representation of the overall population served by DLL is likely to produce relatively representative information about both the costs and the impacts of the program. However, the generalizability of our estimates also is limited in the sense that the DLL programs that we studied were mature adoptions. Future work should consider the costs and impacts of new or more recently adopted DLL programs, which may yield somewhat higher initial costs and potentially lower initial achievement impacts.</p> <hd id="AN0187023105-31">Conclusion</hd> <p>One-to-one tutoring programs often have the perception of being expensive interventions, but our results suggest that the DLL program is relatively cost-effective compared to another Spanish-language literacy intervention, <emph>Exito Para Todos</emph>, and is comparable to other early elementary English-language literacy interventions. In addition, DLL features the key components typically cited as those characterizing the most effective tutoring programs. As Robinson et al. ([<reflink idref="bib50" id="ref107">50</reflink>]) suggested, effective tutoring should be delivered three or more times per week by well-trained teachers. The programs should, further, emphasize data use to inform student progress and tailor instruction. They should use high-quality instructional materials aligned with classroom content, allowing the tutors to reinforce regular classroom instruction. Finally, by having DLL students work with the same caring tutor for 12–20 weeks, trusting student-teacher relationships can develop and facilitate stronger understandings of each student's specific learning needs.</p> <p>In the wake of COVID-19 school closures and lost learning time, there have been numerous calls for increasing the use of tutoring programs to help the many students who have fallen behind get back on track (Robinson et al., [<reflink idref="bib50" id="ref108">50</reflink>]; Slavin, [<reflink idref="bib55" id="ref109">55</reflink>]). Indeed, the persistent inequities in opportunity experienced by school-aged ELs have certainly been amplified by the COVID-19 pandemic (Hopkins & Weddle, [<reflink idref="bib28" id="ref110">28</reflink>]; Hough, [<reflink idref="bib32" id="ref111">32</reflink>]). Under typical circumstances, ELs face a host of barriers to their education, such as suboptimal learning environments, less experienced and poorly trained teachers, and English-only programming (Gandara & Hopkins, [<reflink idref="bib22" id="ref112">22</reflink>]; NASEM, [<reflink idref="bib43" id="ref113">43</reflink>]; Suárez-Orozco et al., [<reflink idref="bib59" id="ref114">59</reflink>]). Families with young children entering kindergarten and first grade in the coming years will have experienced additional hardships because the COVID-19 pandemic further threatened their children's school readiness. Latinx and immigrant families, in particular, have experienced disproportionate impacts on their financial stability, access to public assistance, physical and mental health, and social-emotional well-being (Zamarripa & Roque, [<reflink idref="bib69" id="ref115">69</reflink>]). With these historical and current barriers to school opportunities and success that Latinx ELs are now facing, effective Spanish-language literacy supports (e.g., DLL) will be vital in the coming years.</p> <hd id="AN0187023105-32">Open Research Statements</hd> <p>This manuscript was not required to disclose open research practices, as it was initially submitted prior to JREE mandating open research statements in April 2022.</p> <hd id="AN0187023105-33">Disclosure Statement</hd> <p>No potential conflict of interest was reported by the author(s).</p> <hd id="AN0187023105-34">Appendix A</hd> <p>Table A1. Amortization periods for DLL materials and facilities.</p> <p> <ephtml> <table><thead><tr><td>Ingredient type</td><td>Input</td><td>Amortization period (years)</td></tr></thead><tbody valign="top"><tr><td>Materials</td><td>DLL textbooks (for PD)</td><td char=".">5</td></tr><tr><td>Materials</td><td>DLL starter kit (for direct instruction)</td><td char=".">5</td></tr><tr><td>Materials</td><td>Miscellaneous professional materials</td><td char=".">5</td></tr><tr><td>Facilities</td><td>Classroom for training (PD)</td><td char=".">30</td></tr><tr><td>Facilities</td><td>School district office</td><td char=".">30</td></tr><tr><td>Facilities</td><td>One-way glass room</td><td char=".">30</td></tr><tr><td>Facilities</td><td>Classroom for PD</td><td char=".">30</td></tr><tr><td>Other inputs</td><td>Training course tuition</td><td char=".">10</td></tr><tr><td>Other inputs</td><td>Transportation for PD</td><td char=".">10</td></tr><tr><td>Other inputs</td><td>Conference registration and transportation</td><td char=".">10</td></tr></tbody></table> </ephtml> </p> <p>9 <emph>Note.</emph> DLL: Descubriendo la Lectura; PD: professional development. Only the startup training and the initial conference costs are amortized. Source: University trainers' interviews and Hollands et al. ([<reflink idref="bib31" id="ref116">31</reflink>]).</p> <ref id="AN0187023105-35"> <title> Notes </title> <blist> <bibl id="bib1" idref="ref5" type="bt">1</bibl> <bibtext> See https://ies.ed.gov/seer/index.asp.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref8" type="bt">2</bibl> <bibtext> See https://readingrecovery.org/.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref33" type="bt">3</bibl> <bibtext> The full multicohort impact study includes information from three school years (i.e., 2016–17, 2017–18, and 2018–19) and additional schools and districts not included in the 2017–18 cost and impact study samples. The achievement impacts across the three school years were consistent, but we link the impact estimates from the second cohort only to mirror the cost analysis sample.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref39" type="bt">4</bibl> <bibtext> For instance, in all study districts, rooms with one-way glass were used by other programs such as RR, and some sites reported already having access to a one-way glass room. We apportioned the cost of the one-way glass rooms to reflect only what portion of time was exclusively for the DLL intervention.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref7" type="bt">5</bibl> <bibtext> The formula to calculate the annualization factor used to spread the cost of a resource used for multiple years was</bibtext> </blist> <blist> <bibtext>Graph</bibtext> </blist> <blist> <bibtext> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mi>a</mi><mo stretchy="true">(</mo><mrow><mi>r</mi><mo>,</mo><mi>n</mi></mrow><mo stretchy="true">)</mo><mo>=</mo><mrow><mfrac><mrow><mo stretchy="true">(</mo><mrow><mi>r</mi><mrow><msup><mrow><mo stretchy="false">(</mo><mn>1</mn><mo>+</mo><mi>r</mi><mo stretchy="false">)</mo></mrow><mrow><mi>n</mi></mrow></msup></mrow></mrow><mo stretchy="true">)</mo></mrow><mrow><mo stretchy="true">(</mo><mrow><mrow><msup><mrow><mo stretchy="false">(</mo><mn>1</mn><mo>+</mo><mi>r</mi><mo stretchy="false">)</mo></mrow><mrow><mi>n</mi></mrow></msup></mrow><mo>−</mo><mn>1</mn></mrow><mo stretchy="true">)</mo></mrow></mfrac></mrow></math> </ephtml> , where <emph>r</emph> is the interest rate and <emph>n</emph> is the number of years over which the resource remains useful (Levin et al., [37]). We assumed a 3% interest rate for all nonpersonnel resources (Levin et al., [37]).</bibtext> </blist> <blist> <bibl id="bib6" idref="ref42" type="bt">6</bibl> <bibtext> A district's count of the number of students participating in the DLL program was based on the number of students identified by IDEC. This count differed from the number of students randomly assigned to the DLL program and included in the impact study.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref46" type="bt">7</bibl> <bibtext> The IDEC data revealed 199 students served during the 2017–18 school year. The primary reason for this discrepancy is that several DLL students were tested and deemed eligible for services after the impact study's randomization process, which occurred at the start of the school year.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref51" type="bt">8</bibl> <bibtext> See https://charts.intensiveintervention.org/screening/tool/?id=77c5c64492268897).</bibtext> </blist> <blist> <bibl id="bib9" idref="ref43" type="bt">9</bibl> <bibtext> See https://ies.ed.gov/ncee/wwc.</bibtext> </blist> <blist> <bibtext> The studies conducted by Dianda and Flaherty ([19]) and Nunnery et al. ([47]) used the Woodcock-Johnson assessment, which includes three subtests: Word Identification, Word Attack, and Passage Comprehension. Similar to the <emph>Logramos</emph> and <emph>IDO</emph> outcomes we report here, these subtests were used to create a composite score reflecting a range of Spanish-language outcomes within the overall literacy domain.</bibtext> </blist> <blist> <bibtext> The cited study by Borman et al. ([8]) did, incidentally, include resources delivered to schools that went beyond the targeted Grades K–2. Specifically, all students in Grades K–5 could benefit from the <emph>schoolwide resources</emph>, such as the parent outreach program and the schoolwide solutions team. Further, during each year of this 3-year study, the Success for All team began delivering <emph>instructional resources</emph> to the next grade level beyond the targeted K–2 grades. Specifically, beginning in Year 2 of the study, Grade 3 students began their participation, in Year 3 Grade 4 students began participating, and after the final year of the study in Year 4, all students in Grades K–5 were served by the targeted instructional components.</bibtext> </blist> <blist> <bibtext> See https://<ulink href="http://www.idecweb.us/publications.aspx">www.idecweb.us/publications.aspx</ulink>.</bibtext> </blist> </ref> <ref id="AN0187023105-36"> <title> References </title> <blist> <bibtext> August, D., McCardle, P., & Shanahan, T. (2014). Developing literacy in English language learners: Findings from a review of the experimental research. School Psychology Review, 43 (4), 490 – 498. https://doi.org/10.1080/02796015.2014.12087417</bibtext> </blist> <blist> <bibtext> August, D., & Shanahan, T. (2006). Developing literacy in second-language learners: Report of the National Literacy Panel on language-minority children and youth. Lawrence Erlbaum.</bibtext> </blist> <blist> <bibtext> Baird, M. D., & Pane, J. F. (2019). Translating standardized effects of education programs into more interpretable metrics. Educational Researcher, 48 (4), 217 – 228. https://doi.org/10.3102/0013189X19848729</bibtext> </blist> <blist> <bibtext> Baker, S., Geva, E., Kieffer, M. J., Lesaux, N., Linan-Thompson, S., Morris, J., Proctor, C. P., Russel, R., Gersten, R., Dimino, J., Jayanthi, M., Haymond, K., & Newman-Gonchar, R. (2014). Teaching academic content and literacy to English learners in elementary and middle school (NCEE 2014-4012). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/ncee/wwc/Docs/PracticeGuide/english_learners_pg_040114.pdf</bibtext> </blist> <blist> <bibtext> Belfield, C. R., & Bowden, B. A. (2018). Using resource and cost considerations to support educational evaluation: Six domains. Educational Researcher, 48 (2), 120 – 127. https://doi.org/10.3102/0013189X18814447</bibtext> </blist> <blist> <bibtext> Belfield, C., Bowden, A. B., Klapp, A., Levin, H., Shand, R., & Zander, S. (2015). The economic value of social and emotional learning. Journal of Benefit-Cost Analysis, 6 (3), 508 – 544. https://doi.org/10.1017/bca.2015.55</bibtext> </blist> <blist> <bibtext> Borman, G. D., & Hewes, G. (2002). The long-term effects and cost-effectiveness of Success for All. Educational Evaluation and Policy Analysis, 24 (4), 243 – 266. https://doi.org/10.3102/01623737024004243</bibtext> </blist> <blist> <bibtext> Borman, G. D., Slavin, R. E., Cheung, A., Chamberlain, A., Madden, N., & Chambers, B. (2007). Final reading outcomes of the national randomized field trial of Success for All. American Educational Research Journal, 44 (3), 701 – 731. https://doi.org/10.3102/0002831207306743</bibtext> </blist> <blist> <bibtext> Borman, T. H., Borman, G. D., Houghton, S., Park, S.-J., Zhu, B., Martin, A., & Wilkinson-Flicker, S. (2019). Addressing literacy needs of struggling Spanish-speaking first graders: First-year results from a national randomized controlled trial of Descubriendo la Lectura. AERA Open, 5 (3), 233285841987048. https://doi.org/10.1177/2332858419870488</bibtext> </blist> <blist> <bibtext> Borman, T. H., Borman, G. D., Zhu, B., Park, S.-J., Houghton, S., & Martin, A. (2023). The Spanish- and English-language literacy impacts of Descubriendo la Lectura across three experimental replications. Manuscript in press.</bibtext> </blist> <blist> <bibtext> Calderón, M., Slavin, R., & Sánchez, M. (2011). Effective instruction for English learners. The Future of Children, 21 (1), 103 – 127. https://files.eric.ed.gov/fulltext/EJ920369.pdf https://doi.org/10.1353/foc.2011.0007</bibtext> </blist> <blist> <bibtext> Chambers, J. (1999). Measuring resources in education: From accounting to the Resource Cost Model approach (NCES-WP-1999-16). U.S. Department of Education, National Center for Education Statistics. https://files.eric.ed.gov/fulltext/ED433613.pdf</bibtext> </blist> <blist> <bibtext> Cheung, A., & Slavin, R. E. (2005). Effective reading programs for English language learners and other language-minority students. Bilingual Research Journal, 29 (2), 241 – 267. https://doi.org/10.1080/15235882.2005.10162835</bibtext> </blist> <blist> <bibtext> Cheung, A. C., & Slavin, R. E. (2012). Effective reading programs for Spanish-dominant English language learners (ELLs) in the elementary grades: A synthesis of research. Review of Educational Research, 82 (4), 351 – 395. https://doi.org/10.3102/0034654312465472</bibtext> </blist> <blist> <bibtext> Clay, M. M. (1993). An observation survey of early literacy achievement. Heinemann.</bibtext> </blist> <blist> <bibtext> Clay, M. M. (2005). Literacy lessons designed for individuals. Heinemann.</bibtext> </blist> <blist> <bibtext> Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Academic Press.</bibtext> </blist> <blist> <bibtext> Cornman, S. Q., Nixon, L. C., Spence, M. J., Taylor, L. L., & Geverdt, D. E. (2019). Education Demographic and Geographic Estimates (EDGE) Program: American Community Survey Comparable Wage Index for Teachers (ACS-CWIFT) (NCES 2018-130). U.S. Department of Education, National Center for Education Statistics. https://nces.ed.gov/programs/edge/docs/EDGE_ACS_CWIFT_FILEDOC.pdf</bibtext> </blist> <blist> <bibtext> Dianda, M., & Flaherty, J. (1995). Effects of Success for All on the reading achievement of first graders in California bilingual programs [Paper presentation]. Annual Meeting of the American Educational Research Association, San Francisco, CA, United States.</bibtext> </blist> <blist> <bibtext> Escamilla, K., Andrade, A., Basurto, A., & Ruiz, O. (1996). Instrumento de observación de los logros de la lecto-escritura inicial. Heinemann.</bibtext> </blist> <blist> <bibtext> Francis, D., Lesaux, N. K., & August, D. (2006). Language of instruction. In D. L. August & T. Shanahan (Eds.), Developing literacy in a second language: Report of the National Literacy Panel (pp. 365 – 410). Lawrence Erlbaum.</bibtext> </blist> <blist> <bibtext> Gandara, P., & Hopkins, M. (Eds.). (2010). Forbidden language: English learners and restrictive language policies. Teachers College Press.</bibtext> </blist> <blist> <bibtext> Gersten, R., Baker, S. K., Shanahan, T., Linan-Thompson, S., Collins, P., & Scarcella, R. (2007). Effective literacy and English language instruction for English learners in the elementary grades (NCEE 2007-4011). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/ncee/wwc/Docs/PracticeGuide/20074011.pdf</bibtext> </blist> <blist> <bibtext> Greene, J. (1997). A meta-analysis of the Rossell and Baker review of bilingual education research. Bilingual Research Journal, 21 (2-3), 103 – 122. https://doi.org/10.1080/15235882.1997.10668656</bibtext> </blist> <blist> <bibtext> Harris, D. N. (2009). Toward policy-relevant benchmarks for interpreting effect sizes: Combining effects with costs. Educational Evaluation and Policy Analysis, 31 (1), 3 – 29. https://doi.org/10.3102/0162373708327524</bibtext> </blist> <blist> <bibtext> Hartman, W. T., Bolton, D., & Monk, D. (2001). A synthesis of two approaches to school-level financial data: The accounting and resource cost model approaches.. In William J. Fowler, Jr. (Ed.), Selected Papers in School Finance, 2000–01 (NCES 2001–378). U.S. Department of Education, National Center for Education Statistics. https://nces.ed.gov/pubs2001/2001378.pdf</bibtext> </blist> <blist> <bibtext> Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2008). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2 (3), 172 – 177. https://doi.org/10.1111/j.1750-8606.2008.00061.x</bibtext> </blist> <blist> <bibtext> Hopkins, M., & Weddle, H. (2020). Restart and recovery: Access and equity for English learner students and families during COVID-19: Recommendations for state leaders. Council of Chief State School Officers. https://ccsso.org/sites/default/files/2020-10/CCSSO_Restart_%26_Recovery_Access_and_Equity_for_English_Learner_v3.pdf</bibtext> </blist> <blist> <bibtext> Hollands, F. M., Hanisch-Cerda, B., Levin, H. M., Belfield, C. R., Menon, A., Shand, R., Pan, Y., Bakir, I., & Cheng, H. (2015). CostOut—The CBCSE cost tool kit. Teachers College, Columbia University, Center for Benefit-Cost Studies of Education.</bibtext> </blist> <blist> <bibtext> Hollands, F. M., Pan, Y., Shand, R., Cheng, H., Levin, H. M., Belfield, C. R., Kieffer, M., Bowden, A. B., & Hanisch-Cerda, B. (2013). Improving early literacy: Cost-effectiveness analysis of effective reading programs. Teachers College, Columbia University, Center for Benefit-Cost Studies of Education. <ulink href="http://frg.vkcsites.org/wp-content/uploads/2018/07/KPALS-PDF-Improving-Early-Literacy.pdf">http://frg.vkcsites.org/wp-content/uploads/2018/07/KPALS-PDF-Improving-Early-Literacy.pdf</ulink></bibtext> </blist> <blist> <bibtext> Hollands, F. M., Pratt-Williams, J., & Shand, R. (2021). Cost analysis standards & guidelines 1.1. Cost Analysis in Practice (CAP) project. https://static1.squarespace.com/static/5eb0d7c7df68b75104fbc784/t/63338376b384231897ba8b38/1664320376924/CAP+Project+Cost+Analysis+Guidelines_1.1_Final.pdf</bibtext> </blist> <blist> <bibtext> Hough, H. J. (2021). COVID-19, the educational equity crisis, and the opportunity ahead. Brookings Institution. https://<ulink href="http://www.brookings.edu/blog/brown-center-chalkboard/2021/04/29/covid-19-the-educational-equity-crisis-and-the-opportunity-ahead/">www.brookings.edu/blog/brown-center-chalkboard/2021/04/29/covid-19-the-educational-equity-crisis-and-the-opportunity-ahead/</ulink></bibtext> </blist> <blist> <bibtext> King, J. A. (1994). Meeting the educational needs of at-risk students: A cost analysis of three models. Educational Evaluation and Policy Analysis, 16 (1), 1 – 19. https://doi.org/10.3102/01623737016001001</bibtext> </blist> <blist> <bibtext> Kraft, M. A. (2020). Interpreting effect sizes of education interventions. Educational Researcher, 49 (4), 241 – 253. https://doi.org/10.3102/0013189X20912798</bibtext> </blist> <blist> <bibtext> Levin, H. M. (2001). Waiting for Godot: Cost-effectiveness analysis in education. New Directions for Evaluation, 2001 (90), 55 – 68. https://doi.org/10.1002/ev.12</bibtext> </blist> <blist> <bibtext> Levin, H. M., & Belfield, C. (2015). Guiding the development and use of cost-effectiveness analysis in education. Journal of Research on Educational Effectiveness, 8 (3), 400 – 418. https://doi.org/10.1080/19345747.2014.915604</bibtext> </blist> <blist> <bibtext> Levin, H., McEwan, P., Belfield, C., Bowden, A. B., & Shand, R. (2018). Economic evaluation in education: Cost effectiveness and benefit-cost analysis (3rd ed.). Sage Publications.</bibtext> </blist> <blist> <bibtext> Logramos. (2006). 2005 norms and score conversions with technical information. Riverside. https://<ulink href="http://www.riversideinsights.com/log%5ftercera">www.riversideinsights.com/log%5ftercera</ulink></bibtext> </blist> <blist> <bibtext> Lipsey, M. W., Puzio, K., Yun, C., Hebert, M. A., Steinka-Fry, K., Cole, M. W., Roberts, M., Anthony, K. S., & Busick, M. D. (2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms. U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research. https://ies.ed.gov/ncser/pubs/20133000/pdf/20133000.pdf</bibtext> </blist> <blist> <bibtext> Ludwig, C., Guo, K., & Georgiou, G. K. (2019). Are reading interventions for English language learners effective? A meta-analysis. Journal of Learning Disabilities, 52 (3), 220 – 231. https://doi.org/10.1177/0022219419825855</bibtext> </blist> <blist> <bibtext> May, H., Gray, A., Gillespie, J., Sirinides, P., Sam, C., Goldsworthy, H., Armijo, M., & Tognatta, N. (2013). Evaluation of the i3 scale-up of Reading Recovery: Year one report, 2011–12 (CPRE Research Report No. RR-76). Consortium for Policy Research in Education. https://<ulink href="http://www.cpre.org/sites/default/files/researchreport/1488%5freadingrecoveryreport.pdf">www.cpre.org/sites/default/files/researchreport/1488%5freadingrecoveryreport.pdf</ulink></bibtext> </blist> <blist> <bibtext> McField, G., & McField, D. (2014). The consistent outcome of bilingual education programs: A meta-analysis of meta-analyses. In G. McField (Ed.), The miseducation of English learners (pp. 267 – 299). Information Age.</bibtext> </blist> <blist> <bibtext> National Academies of Sciences, Engineering, and Medicine (NASEM). (2017). Promoting the educational success of children and youth learning English: Promising futures. National Academies Press.</bibtext> </blist> <blist> <bibtext> National Center for Education Statistics. (2018). Locale classifications. U.S. Department of Education, Institute of Education Sciences. https://nces.ed.gov/programs/edge/Geographic/LocaleBoundaries</bibtext> </blist> <blist> <bibtext> National Center for Education Statistics. (2020a). The condition of education 2020 (NCES 2020-144). U.S. Department of Education, Institute of Education Sciences. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2020144</bibtext> </blist> <blist> <bibtext> National Center for Education Statistics. (2020b). Revenues and expenditures for public elementary and secondary education: FY 18 (NCES 2020-306). U.S. Department of Education, Institute of Education Sciences. https://nces.ed.gov/pubs2020/2020306.pdf</bibtext> </blist> <blist> <bibtext> Nunnery, J., Slavin, R., Ross, S., Smith, L., Hunter, P., & Stubbs, J. (1997). Effects of full and partial implementations of Success for All on student reading achievement in English and Spanish [Paper presentation]. Annual Meeting of the American Educational Research Association, Chicago, IL, United States.</bibtext> </blist> <blist> <bibtext> Reading Recovery Council of North America (RRCNA). (2018). Standards and guidelines of Reading Recovery in the United States (8th ed.). https://readingrecovery.org/wp-content/uploads/2018/12/SG2018-Master-Full.pdf</bibtext> </blist> <blist> <bibtext> Richards-Tutor, C., Baker, D. L., Gersten, R., Baker, S. K., & Smith, J. M. (2016). The effectiveness of reading interventions for English learners: A research synthesis. Exceptional Children, 82 (2), 144 – 169. https://doi.org/10.1177/0014402915585483</bibtext> </blist> <blist> <bibtext> Robinson, C. D., Kraft, M. A., Loeb, S., & Schueler, B. E. (2021). Accelerating student learning with high-dosage tutoring. EdResearch for Recovery. https://annenberg.brown.edu/sites/default/files/EdResearch_for_Recovery_Design_Principles_1.pdf</bibtext> </blist> <blist> <bibtext> Rolstad, K., Mahoney, K., & Glass, G. (2005). The big picture: A meta-analysis of program effectiveness research on English language learners. Educational Policy, 19 (4), 572 – 594. https://doi.org/10.1177/0895904805278067</bibtext> </blist> <blist> <bibtext> Shrestha, P., Tracy, T., Mazal, M., Blakeney, A., Kennedy, N., & May, H. (2022). A cost analysis of Reading Recovery and alternate interventions under the i3 scale-up [Paper presentation]. Annual Meeting of the American Education Research Association, San Diego, CA, United States.</bibtext> </blist> <blist> <bibtext> Simon, J. (2011). A cost-effectiveness analysis of early literacy interventions [Unpublished doctoral dissertation]. Columbia University.</bibtext> </blist> <blist> <bibtext> Sirinides, P., Gray, A., & May, H. (2018). The impacts of Reading Recovery at scale: Results from the 4-year i3 external evaluation. Educational Evaluation and Policy Analysis, 40 (3), 316 – 335. https://doi.org/10.3102/0162373718764828</bibtext> </blist> <blist> <bibtext> Slavin, R. (2021). Launching ProvenTutoring. https://robertslavinsblog.wordpress.com/2021/04/26/launching-proventutoring/</bibtext> </blist> <blist> <bibtext> Slavin, R. E., & Cheung, A. (2004). How do English language learners learn to read? Educational Leadership, 61 (6), 52 – 57. https://files.ascd.org/staticfiles/ascd/pdf/journals/ed_lead/el200403_slavin.pdf</bibtext> </blist> <blist> <bibtext> Snyder, E., Witmer, S. E., & Schmitt, H. (2017). English language learners and reading instruction: A review of the literature. Preventing School Failure: Alternative Education for Children and Youth, 61 (2), 136 – 145. https://doi.org/10.1080/1045988X.2016.1219301</bibtext> </blist> <blist> <bibtext> Sparks, S. D. (2019). More education studies look at cost-effectiveness. Education Week, April 9. https://<ulink href="http://www.edweek.org/leadership/more-education-studies-look-at-cost-effectiveness/2019/04">www.edweek.org/leadership/more-education-studies-look-at-cost-effectiveness/2019/04</ulink></bibtext> </blist> <blist> <bibtext> Suárez-Orozco, C., Gaytán, F. X., Bang, H. J., Pakes, J., O'Connor, E., & Rhodes, J. (2010). Academic trajectories of newcomer immigrant youth. Developmental Psychology, 46 (3), 602 – 618. https://doi.org/10.1037/a0018201</bibtext> </blist> <blist> <bibtext> Umansky, I., & Reardon, S. F. (2014). Reclassification patterns among Latino English learner students in bilingual, dual immersion, and English immersion classrooms. American Educational Research Journal, 51 (5), 879 – 912. https://doi.org/10.3102/0002831214545110</bibtext> </blist> <blist> <bibtext> U.S. Department of Labor. (n.d). CPI inflation calculator. https://data.bls.gov/cgi-bin/cpicalc.pl</bibtext> </blist> <blist> <bibtext> Valentino, R. A., & Reardon, S. F. (2015). Effectiveness of four instructional programs designed to serve English learners: Variation by ethnicity and initial English proficiency. Educational Evaluation and Policy Analysis, 37 (4), 612 – 637. https://doi.org/10.3102/0162373715573310</bibtext> </blist> <blist> <bibtext> Vaughn, S., Cirino, P. T., Linan-Thompson, S., Mathes, P. G., Carlson, C. D., Hagan, E. C., Pollard-Durodola, S. D., Fletcher, J. M., & Francis, D. J. (2006). Effectiveness of a Spanish intervention and an English intervention for English-language learners at risk for reading problems. American Educational Research Journal, 43 (3), 449 – 487. https://doi.org/10.3102/00028312043003449</bibtext> </blist> <blist> <bibtext> Vaughn, S., Linan-Thompson, S., Mathes, P. G., Cirino, P. T., Carlson, C. D., Pollard-Durodola, S. D., Cardenas-Hagan, E., & Francis, D. J. (2006). Effectiveness of Spanish intervention for first-grade English language learners at risk for reading difficulties. Journal of Learning Disabilities, 39 (1), 56 – 73. https://doi.org/10.1177/00222194060390010601</bibtext> </blist> <blist> <bibtext> What Works Clearinghouse (WWC). (2006). Intervention report: Enhanced Proactive Reading. U.S. Department of Education, Institute of Education Sciences. https://ies.ed.gov/ncee/wwc/Docs/InterventionReports/WWC_Proactive_Reading092806.pdf</bibtext> </blist> <blist> <bibtext> What Works Clearinghouse (WWC). (2022). What Works Clearinghouse procedures and standards handbook (version 5.0). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/ncee/wwc/Handbooks</bibtext> </blist> <blist> <bibtext> Willig, A. (1985). A meta-analysis of selected studies on the effectiveness of bilingual education. Review of Educational Research, 55 (3), 269 – 317. https://doi.org/10.3102/00346543055003269</bibtext> </blist> <blist> <bibtext> Yeh, S. (2010). The cost effectiveness of 22 approaches for raising student achievement. Journal of Education Finance, 36 (1), 38 – 75. https://<ulink href="http://www.jstor.org/stable/40704405">www.jstor.org/stable/40704405</ulink> https://doi.org/10.1353/jef.0.0029</bibtext> </blist> <blist> <bibtext> Zamarripa, R., & Roque, L. (2021). Latinos face disproportionate health and economic impacts from COVID-19. Center for American Progress. https://<ulink href="http://www.americanprogress.org/issues/economy/reports/2021/03/05/496733/latinos-face-disproportionate-health-economic-impacts-covid-19/">www.americanprogress.org/issues/economy/reports/2021/03/05/496733/latinos-face-disproportionate-health-economic-impacts-covid-19/</ulink></bibtext> </blist> </ref> <aug> <p>By Geoffrey D. Borman; Iliana Brodziak de los Reyes; Trisha H. Borman; Scott Houghton; So Jung Park; Bo Zhu and Alejandra Martin</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib45" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib35" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib36" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib58" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib66" firstref="ref6"></nolink> <nolink nlid="nl6" bibid="bib48" firstref="ref9"></nolink> <nolink nlid="nl7" bibid="bib11" firstref="ref10"></nolink> <nolink nlid="nl8" bibid="bib54" firstref="ref12"></nolink> <nolink nlid="nl9" bibid="bib43" firstref="ref16"></nolink> <nolink nlid="nl10" bibid="bib60" firstref="ref17"></nolink> <nolink nlid="nl11" bibid="bib62" firstref="ref18"></nolink> <nolink nlid="nl12" bibid="bib21" firstref="ref19"></nolink> <nolink nlid="nl13" bibid="bib24" firstref="ref20"></nolink> <nolink nlid="nl14" bibid="bib42" firstref="ref21"></nolink> <nolink nlid="nl15" bibid="bib51" firstref="ref22"></nolink> <nolink nlid="nl16" bibid="bib56" firstref="ref23"></nolink> <nolink nlid="nl17" bibid="bib67" firstref="ref24"></nolink> <nolink nlid="nl18" bibid="bib41" firstref="ref27"></nolink> <nolink nlid="nl19" bibid="bib37" firstref="ref29"></nolink> <nolink nlid="nl20" bibid="bib12" firstref="ref31"></nolink> <nolink nlid="nl21" bibid="bib26" firstref="ref32"></nolink> <nolink nlid="nl22" bibid="bib44" firstref="ref34"></nolink> <nolink nlid="nl23" bibid="bib18" firstref="ref36"></nolink> <nolink nlid="nl24" bibid="bib46" firstref="ref37"></nolink> <nolink nlid="nl25" bibid="bib29" firstref="ref38"></nolink> <nolink nlid="nl26" bibid="bib61" firstref="ref41"></nolink> <nolink nlid="nl27" bibid="bib10" firstref="ref44"></nolink> <nolink nlid="nl28" bibid="bib38" firstref="ref45"></nolink> <nolink nlid="nl29" bibid="bib20" firstref="ref47"></nolink> <nolink nlid="nl30" bibid="bib15" firstref="ref49"></nolink> <nolink nlid="nl31" bibid="bib16" firstref="ref50"></nolink> <nolink nlid="nl32" bibid="bib17" firstref="ref55"></nolink> <nolink nlid="nl33" bibid="bib27" firstref="ref57"></nolink> <nolink nlid="nl34" bibid="bib34" firstref="ref58"></nolink> <nolink nlid="nl35" bibid="bib39" firstref="ref59"></nolink> <nolink nlid="nl36" bibid="bib30" firstref="ref62"></nolink> <nolink nlid="nl37" bibid="bib25" firstref="ref63"></nolink> <nolink nlid="nl38" bibid="bib68" firstref="ref66"></nolink> <nolink nlid="nl39" bibid="bib52" firstref="ref67"></nolink> <nolink nlid="nl40" bibid="bib40" firstref="ref70"></nolink> <nolink nlid="nl41" bibid="bib49" firstref="ref71"></nolink> <nolink nlid="nl42" bibid="bib13" firstref="ref75"></nolink> <nolink nlid="nl43" bibid="bib14" firstref="ref76"></nolink> <nolink nlid="nl44" bibid="bib23" firstref="ref77"></nolink> <nolink nlid="nl45" bibid="bib57" firstref="ref78"></nolink> <nolink nlid="nl46" bibid="bib65" firstref="ref80"></nolink> <nolink nlid="nl47" bibid="bib64" firstref="ref81"></nolink> <nolink nlid="nl48" bibid="bib63" firstref="ref82"></nolink> <nolink nlid="nl49" bibid="bib19" firstref="ref83"></nolink> <nolink nlid="nl50" bibid="bib47" firstref="ref85"></nolink> <nolink nlid="nl51" bibid="bib33" firstref="ref88"></nolink> <nolink nlid="nl52" bibid="bib53" firstref="ref92"></nolink> <nolink nlid="nl53" bibid="bib50" firstref="ref107"></nolink> <nolink nlid="nl54" bibid="bib55" firstref="ref109"></nolink> <nolink nlid="nl55" bibid="bib28" firstref="ref110"></nolink> <nolink nlid="nl56" bibid="bib32" firstref="ref111"></nolink> <nolink nlid="nl57" bibid="bib22" firstref="ref112"></nolink> <nolink nlid="nl58" bibid="bib59" firstref="ref114"></nolink> <nolink nlid="nl59" bibid="bib69" firstref="ref115"></nolink> <nolink nlid="nl60" bibid="bib31" firstref="ref116"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1502025
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: The Costs and Impacts of 'Descubriendo la Lectura': Evidence from a Multisite Experimental Study
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Geoffrey+D%2E+Borman%22">Geoffrey D. Borman</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-7039-8208">0000-0002-7039-8208</externalLink>)<br /><searchLink fieldCode="AR" term="%22Iliana+Brodziak+de+los+Reyes%22">Iliana Brodziak de los Reyes</searchLink><br /><searchLink fieldCode="AR" term="%22Trisha+H%2E+Borman%22">Trisha H. Borman</searchLink><br /><searchLink fieldCode="AR" term="%22Scott+Houghton%22">Scott Houghton</searchLink><br /><searchLink fieldCode="AR" term="%22So+Jung+Park%22">So Jung Park</searchLink><br /><searchLink fieldCode="AR" term="%22Bo+Zhu%22">Bo Zhu</searchLink><br /><searchLink fieldCode="AR" term="%22Alejandra+Martin%22">Alejandra Martin</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Research+on+Educational+Effectiveness%22"><i>Journal of Research on Educational Effectiveness</i></searchLink>. 2025 18(3):739-768.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 30
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305A160060
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Early+Childhood+Education%22">Early Childhood Education</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+1%22">Grade 1</searchLink><br /><searchLink fieldCode="EL" term="%22Primary+Education%22">Primary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Elementary+Schools%22">Elementary Schools</searchLink><br /><searchLink fieldCode="DE" term="%22Grade+1%22">Grade 1</searchLink><br /><searchLink fieldCode="DE" term="%22Intervention%22">Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22English+Learners%22">English Learners</searchLink><br /><searchLink fieldCode="DE" term="%22Spanish+Speaking%22">Spanish Speaking</searchLink><br /><searchLink fieldCode="DE" term="%22Literacy%22">Literacy</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Achievement%22">Reading Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Difficulties%22">Reading Difficulties</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Implementation%22">Program Implementation</searchLink><br /><searchLink fieldCode="DE" term="%22Expenditure+per+Student%22">Expenditure per Student</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Finance%22">Educational Finance</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Costs%22">Program Costs</searchLink><br /><searchLink fieldCode="DE" term="%22School+Districts%22">School Districts</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Arizona%22">Arizona</searchLink><br /><searchLink fieldCode="DE" term="%22California%22">California</searchLink><br /><searchLink fieldCode="DE" term="%22Illinois%22">Illinois</searchLink><br /><searchLink fieldCode="DE" term="%22Texas%22">Texas</searchLink><br /><searchLink fieldCode="DE" term="%22Wisconsin%22">Wisconsin</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/19345747.2024.2358824
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1934-5747<br />1934-5739
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: We estimate costs and impacts of "Descubriendo la Lectura" (DLL), an intervention designed to improve the literacy skills of Spanish-speaking first graders struggling with reading. Collecting cost data from 24 schools in seven districts across four states participating in a multisite randomized controlled trial of DLL, we used the ingredients method to identify all personnel and nonpersonnel resources and then assign costs to each ingredient. The average cost per student of DLL is approximately $7,120 (in 2022 dollars), with teacher and teacher leader personnel expenditures accounting for more than 90% of the total costs. Impact analyses from the student-level random assignment study suggest DLL has substantial effects on Spanish-language literacy achievement, with effect sizes from d = 0.23 to d = 0.91. Linking estimated costs and achievement impacts, the impact per $1,000 investment in DLL is between d = 0.03 and d = 0.13, and a cost between $7,824 and $30,957 per student is associated with a one standard deviation increase in student achievement. These estimates compare favorably with those found for other interventions with recent cost and impact data.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1502025
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1502025
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/19345747.2024.2358824
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 30
        StartPage: 739
    Subjects:
      – SubjectFull: Elementary Schools
        Type: general
      – SubjectFull: Grade 1
        Type: general
      – SubjectFull: Intervention
        Type: general
      – SubjectFull: English Learners
        Type: general
      – SubjectFull: Spanish Speaking
        Type: general
      – SubjectFull: Literacy
        Type: general
      – SubjectFull: Reading Achievement
        Type: general
      – SubjectFull: Reading Difficulties
        Type: general
      – SubjectFull: Program Implementation
        Type: general
      – SubjectFull: Expenditure per Student
        Type: general
      – SubjectFull: Educational Finance
        Type: general
      – SubjectFull: Program Costs
        Type: general
      – SubjectFull: School Districts
        Type: general
      – SubjectFull: Arizona
        Type: general
      – SubjectFull: California
        Type: general
      – SubjectFull: Illinois
        Type: general
      – SubjectFull: Texas
        Type: general
      – SubjectFull: Wisconsin
        Type: general
    Titles:
      – TitleFull: The Costs and Impacts of 'Descubriendo la Lectura': Evidence from a Multisite Experimental Study
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Geoffrey D. Borman
      – PersonEntity:
          Name:
            NameFull: Iliana Brodziak de los Reyes
      – PersonEntity:
          Name:
            NameFull: Trisha H. Borman
      – PersonEntity:
          Name:
            NameFull: Scott Houghton
      – PersonEntity:
          Name:
            NameFull: So Jung Park
      – PersonEntity:
          Name:
            NameFull: Bo Zhu
      – PersonEntity:
          Name:
            NameFull: Alejandra Martin
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 1934-5747
            – Type: issn-electronic
              Value: 1934-5739
          Numbering:
            – Type: volume
              Value: 18
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Journal of Research on Educational Effectiveness
              Type: main
ResultId 1