View in EDS HTML Full Text PDF Full Text

Exploring Organizational Differences in Perceptions of Implementation Climate and Leadership in Schools: A Mixed Methods Study of Autism EBP Implementation

Saved in:
Bibliographic Details
Title: Exploring Organizational Differences in Perceptions of Implementation Climate and Leadership in Schools: A Mixed Methods Study of Autism EBP Implementation
Language: English
Authors: Melgarejo, Melina, Nahmias, Allison S. (ORCID 0000-0003-4678-0367), Suhrheinrich, Jessica, Schetter, Patricia L., Dean, Michelle, Holt, Tana, Li, Jennica, Stahmer, Aubyn C.
Source: Focus on Autism and Other Developmental Disabilities. Mar 2023 38(1):17-31.
Availability: SAGE Publications and Hammill Institute on Disabilities. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Peer Reviewed: Y
Page Count: 15
Publication Date: 2023
Sponsoring Agency: Institute of Education Sciences (ED)
National Institute of Mental Health (NIMH) (DHHS/NIH)
Contract Number: R324A140005
K01MH109547
Document Type: Journal Articles
Reports - Research
Descriptors: Evidence Based Practice, Outcomes of Education, Autism Spectrum Disorders, Children, Special Education, Leadership, Educational Environment, School Personnel, Attitudes, Program Implementation
DOI: 10.1177/10883576221140149
ISSN: 1088-3576
1538-4829
Abstract: Public programs have been charged with implementation of evidence-based practices (EBPs) to improve outcomes for children with autism. However, research indicates that scale-up of EBPs poses challenges. This study identifies perceived variables linked to effective statewide scale-up of EBPs in special education by exploring implementation climate and leadership across special education organization types (e.g., schools, districts, and regional consortiums). A simultaneous QUAL + QUAN mixed methods design was employed with the primary function of convergence and triangulation. Data were drawn from focus groups with 30 special educators and a statewide survey completed by 656 school personnel in one U.S. state. In general, perceptions of implementation climate and leadership are weak in special education organizations, with strengths at regional levels focused on special education and increased challenges at the school and district levels. Implications for practice and future research are identified.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2023
Accession Number: EJ1370143
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGGAIwKGkKITVA10BhtipSxAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDM_21jFx89EGCuAotQIBEICBmovWmaglnDgz-SRVTbKc6S0XL6ASYQ4k8hH03addAtPvbKlQTkBNYD4eh4UQZiYi5GskQa59jAqqxXCzuLj8nFaU5SECsjFiO5Gt3crMKuC2rqMRXTyp1lBN0NcrZapZUzcR7BsysXKPLVnG5hDZwKUNuK4W9z0SBHN3MM9EfucoPrkVl4tRIV1AjrV80APPz2PpY5ahph--Q10=
Text:
  Availability: 1
  Value: <anid>AN0162352852;fdd01mar.23;2023Mar14.05:21;v2.2.500</anid> <title id="AN0162352852-1">Exploring Organizational Differences in Perceptions of Implementation Climate and Leadership in Schools: A Mixed Methods Study of Autism EBP Implementation </title> <p>Public programs have been charged with implementation of evidence-based practices (EBPs) to improve outcomes for children with autism. However, research indicates that scale-up of EBPs poses challenges. This study identifies perceived variables linked to effective statewide scale-up of EBPs in special education by exploring implementation climate and leadership across special education organization types (e.g., schools, districts, and regional consortiums). A simultaneous QUAL + QUAN mixed methods design was employed with the primary function of convergence and triangulation. Data were drawn from focus groups with 30 special educators and a statewide survey completed by 656 school personnel in one U.S. state. In general, perceptions of implementation climate and leadership are weak in special education organizations, with strengths at regional levels focused on special education and increased challenges at the school and district levels. Implications for practice and future research are identified.</p> <p>Keywords: autism; implementation climate; implementation leadership; evidence-based practices</p> <p>Provision of effective educational services for students with autism is critical. The U.S. Centers for Disease Control estimate that one in 54 children have autism ([<reflink idref="bib19" id="ref1">19</reflink>]), and 710,000 (1.4%) autistic students were served by schools in 2018 ([<reflink idref="bib30" id="ref2">30</reflink>]). Although evidence-based practices (EBPs) for autism have been identified (e.g., [<reflink idref="bib34" id="ref3">34</reflink>]; [<reflink idref="bib42" id="ref4">42</reflink>]), integration of these practices into educational programming has been limited ([<reflink idref="bib14" id="ref5">14</reflink>]; [<reflink idref="bib28" id="ref6">28</reflink>]). Even when teachers are aware of EBPs for autism and attempt to use them, they often have low levels of fidelity (adherence to the EBP protocol; for example, [<reflink idref="bib25" id="ref7">25</reflink>]; [<reflink idref="bib37" id="ref8">37</reflink>]) or limited sustainment after coaching ends ([<reflink idref="bib35" id="ref9">35</reflink>]). Moreover, scaling up—"the deliberate effort to broaden the delivery of an EBP with the intention of reaching larger numbers of a target audience" ([<reflink idref="bib29" id="ref10">29</reflink>])—the use of EBPs across multiple programs, schools, and districts requires navigating additional challenges such as the distributed leadership structure ([<reflink idref="bib17" id="ref11">17</reflink>]; [<reflink idref="bib31" id="ref12">31</reflink>]). Most state systems have very limited capacity for scaling up interventions in ways that lead to meaningful improvements in outcomes for students ([<reflink idref="bib9" id="ref13">9</reflink>]).</p> <hd id="AN0162352852-2">Complexity of Educational Systems and Distributed Leadership</hd> <p>Some potential barriers to the scale-up of EBPs into schools may result from the distributed leadership and complex organizational structures within special education ([<reflink idref="bib17" id="ref14">17</reflink>]; [<reflink idref="bib20" id="ref15">20</reflink>]). School-based services engage teams of professionals, including special and general education teachers, speech/language pathologists, occupational therapists, behavior analysts, psychologists, and paraprofessionals, to support the complex needs of students with autism. Team members have varied teaching philosophies, professional training, and exposure to EBPs and often operate within separate organizational structures. For example, California has over 1,000 school districts distributed among 52 county offices of education (COEs). They are further aggregated into 134 special education local plan areas (SELPAs), regional consortiums responsible for the provisions of special education and compliance with special education laws. The SELPAs collaborate with COE and districts to meet the educational and mental health needs of children. The SELPA staff are often tasked with training and supporting school-based educational teams employed by a district or COE and may be supervised at the school, district, or COE level. The day-to-day delivery of instruction is the responsibility local school districts throughout the state. The COEs are at an intermediate level of the public education system providing a support infrastructure for local schools and districts. Adding to the complexity, many site principals report low confidence in their leadership of special education programming ([<reflink idref="bib16" id="ref16">16</reflink>]). Given this complex leadership structure, it is important to understand key mechanisms associated with effective EBP use to support best outcomes for students with autism.</p> <hd id="AN0162352852-3">Identifying Mechanisms to Support EBP Implementation</hd> <p>Two supporting mechanisms for successful EBP implementation across service sectors are climate and leadership. Implementation climate is the extent to which innovation (or use of new practices) is expected, supported, and rewarded ([<reflink idref="bib40" id="ref17">40</reflink>]). Positive implementation climate has been linked to better EBP sustainment, improved child outcomes, and decreased staff burnout and turn over ([<reflink idref="bib22" id="ref18">22</reflink>]). Implementation leadership, or leader readiness and support for using EBPs, also drives the implementation of EBPs in community programs ([<reflink idref="bib1" id="ref19">1</reflink>], [<reflink idref="bib3" id="ref20">3</reflink>]) and facilitates more effective implementation in mental health services ([<reflink idref="bib5" id="ref21">5</reflink>], [<reflink idref="bib4" id="ref22">4</reflink>]; [<reflink idref="bib13" id="ref23">13</reflink>]; [<reflink idref="bib22" id="ref24">22</reflink>]). When leaders provide clear guidance during implementation and facilitate support among coworkers and from administration for effective implementation, trainees report an increased sense of competence and satisfaction ([<reflink idref="bib11" id="ref25">11</reflink>]).</p> <p>Recent research has explored implementation leadership and climate as mechanisms of change in school-based autism programs. A study of teachers implementing a new EBP for autism indicated that among classrooms with a strong perceived implementation climate, higher fidelity was associated with better student outcomes as measured by an intellectual assessment ([<reflink idref="bib15" id="ref26">15</reflink>]). Leader behavior supporting implementation of EBPs may impact perceived climate. For example, principals' increased frequency of EBP implementation leadership predicted a higher school EBP implementation climate, which in turn predicted higher educator fidelity to one complex autism EBP ([<reflink idref="bib41" id="ref27">41</reflink>]). Melgarejo and colleagues found that leadership behavior was associated with higher staff ratings of implementation climate ([<reflink idref="bib20" id="ref28">20</reflink>]). Positive implementation leadership coupled with the use of support strategies (such as training availability and ongoing monitoring of performance) has been linked to better EBP sustainment and improved child outcomes ([<reflink idref="bib22" id="ref29">22</reflink>]; [<reflink idref="bib32" id="ref30">32</reflink>]).</p> <p>To build upon the existing literature, input from both professionals training educators in EBPs and the broader population of educators who might access the training is key to understanding how EBP implementation can be improved in large public school systems. Thus, we collected information from trainers and a range of educators supporting autistic students to answer our research questions. We capitalized on current efforts to disseminate and implement autism EBP in California through the [<reflink idref="bib7" id="ref31">7</reflink>]; [<reflink idref="bib36" id="ref32">36</reflink>]). CAPTAIN is a statewide initiative that employs interagency collaboration, use of evidence-based training and coaching practices, leader engagement, and data-driven continuous improvement cycles to scale-up the use of EBP ([<reflink idref="bib36" id="ref33">36</reflink>]). More than 200 CAPTAIN members representing education are required to give EBP-specific training and coaching to educators in their regions. This mixed methods study adds to the existing literature by identifying perceived variables linked to effective statewide scale-up of EBPs by exploring implementation climate and leadership across special education organization types using qualitative and quantitative methodology. Specifically, our research questions are as follows:</p> <p></p> <ulist> <item> <bold> Research Question 1: </bold> How is EBP scale-up related to implementation climate and leadership, as reported by trainers (e.g., CAPTAIN cadre)?</item> <p></p> <item> <bold> Research Question 2: </bold> Are these perceptions of implementation climate and leadership shared by educators (e.g., directors, teachers, and paraeducators) in the same educational region and across organizational levels (e.g., schools and districts)?</item> </ulist> <hd id="AN0162352852-4">Method</hd> <p></p> <hd id="AN0162352852-5">Mixed Methods Design and Integration of Qualitative and Quantitative Data</hd> <p>Mixed methods designs use quantitative and qualitative approaches in combination to provide a better understanding of research questions than either approach alone ([<reflink idref="bib24" id="ref34">24</reflink>]; [<reflink idref="bib27" id="ref35">27</reflink>]). A simultaneous QUAL + QUAN mixed methods design was employed. Collection and analysis of quantitative and qualitative data occurred simultaneously and both types of data had equal weight ([<reflink idref="bib24" id="ref36">24</reflink>]). The primary function of the mixed methods analyses was convergence and triangulation. Convergence occurs when both methods are used to answer the same question through comparison of results to see whether they reach the same conclusion (triangulation; [<reflink idref="bib24" id="ref37">24</reflink>]). Integration of qualitative and quantitative data was conducted by first examining the qualitative data to answer Research Question 1: How is EBP scale-up related to implementation climate and leadership, as reported by trainers (e.g., CAPTAIN cadre)? Next, quantitative data were analyzed to answer Research Question 2: Are these perceptions of implementation climate and leadership shared by professionals (e.g., directors and paraeducators) in the same educational region and across organizational levels (e.g., schools and districts)? By including a broader range of participants for the survey, we can investigate whether perceptions are consistent across organizational levels.</p> <hd id="AN0162352852-6">Participants</hd> <p></p> <hd id="AN0162352852-7">Focus Group Sample</hd> <p>Focus group participants were 30 CAPTAIN members (called cadre) representing 28 SELPAs. The longest serving CAPTAIN members were invited to participate in the focus groups. As seen in Table 1, most focus group participants who reported demographic information were behavior specialists or analysts (<emph>n</emph> = 13), female (<emph>n</emph> = 17), and White (<emph>n</emph> = 17), Mean number of years in the field was 16.75 (<emph>SD</emph> = 6.11) and in their current position was 6.14 (<emph>SD</emph> = 5.30). The majority held a master's degree (<emph>n</emph> = 22).</p> <p>Graph</p> <p>Table 1. Frequency Distributions: Demographics of Participants in Study on Autism EBP Implementation.</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /></colgroup><thead><tr><th align="left" rowspan="2">Participant demographics</th><th align="center" colspan="2">Survey</th><th align="center" colspan="2">Focus group</th></tr><tr><th align="center"><italic>n</italic></th><th align="center"><italic>%</italic></th><th align="center"><italic>n</italic></th><th align="center"><italic>%</italic></th></tr></thead><tbody><tr><td colspan="5">Organizational Level</td></tr><tr><td> School</td><td>360</td><td>54.9</td><td>1</td><td>3.3</td></tr><tr><td> District</td><td>198</td><td>30.2</td><td>8</td><td>26.7</td></tr><tr><td> COE</td><td>25</td><td>3.8</td><td>3</td><td>10.0</td></tr><tr><td> SELPA</td><td>73</td><td>11.1</td><td>10</td><td>33.3</td></tr><tr><td> Not reported</td><td>0</td><td>0</td><td>8</td><td>26.7</td></tr><tr><td colspan="5">Job Title</td></tr><tr><td> Special Education Teacher</td><td>222</td><td>33.9</td><td>0</td><td>0</td></tr><tr><td> Specialist (e.g., Behavior, Autism, and Program)</td><td>115</td><td>17.7</td><td>17</td><td>56.7</td></tr><tr><td> Paraprofessional</td><td>67</td><td>10.3</td><td>0</td><td>0</td></tr><tr><td> Psychologist</td><td>59</td><td>9.0</td><td>1</td><td>3.3</td></tr><tr><td> District Admin (SPED Director and Superintendent)</td><td>43</td><td>6.9</td><td>0</td><td>0</td></tr><tr><td> SELPA Director</td><td>24</td><td>3.7</td><td>0</td><td>0</td></tr><tr><td> Program Manager/Coordinator</td><td>17</td><td>2.6</td><td>8</td><td>26.7</td></tr><tr><td> Itinerant Teacher or Teacher on Special Assignment</td><td>24</td><td>3.7</td><td>2</td><td>6.7</td></tr><tr><td> General Education Teacher</td><td>6</td><td>0.9</td><td>0</td><td>0</td></tr><tr><td> DSP (OT, SLP, Counselor, and Social Worker)</td><td>47</td><td>7.3</td><td>0</td><td>0</td></tr><tr><td> School Admin (Principal and Assistant Principal)</td><td>29</td><td>4.5</td><td>0</td><td>0</td></tr><tr><td> Other or not reported</td><td>3</td><td>0.5</td><td>2</td><td>6.7</td></tr><tr><td colspan="5">Education</td></tr><tr><td> High school</td><td>12</td><td>1.8</td><td>0</td><td>0</td></tr><tr><td> Associate degree</td><td>29</td><td>4.4</td><td>0</td><td>0</td></tr><tr><td> Bachelor's degree</td><td>160</td><td>24.4</td><td>3</td><td>10.0</td></tr><tr><td> Master's degree</td><td>427</td><td>65.1</td><td>22</td><td>73.3</td></tr><tr><td> Doctorate</td><td>27</td><td>4.1</td><td>3</td><td>10.0</td></tr><tr><td> Not reported</td><td>1</td><td>0.2</td><td>2</td><td>6.7</td></tr><tr><td colspan="5">Gender</td></tr><tr><td> Female</td><td>500</td><td>76.2</td><td>17</td><td>56.7</td></tr><tr><td> Male</td><td>83</td><td>12.7</td><td>3</td><td>10.0</td></tr><tr><td> Other</td><td>1</td><td>0.2</td><td>0</td><td>0</td></tr><tr><td> Not reported</td><td>72</td><td>11.0</td><td>10</td><td>33.3</td></tr><tr><td colspan="5">Ethnicity</td></tr><tr><td> Hispanic/Latino</td><td>97</td><td>14.8</td><td>2</td><td>6.7</td></tr><tr><td> Not Hispanic/Latino</td><td>470</td><td>71.6</td><td>18</td><td>60.0</td></tr><tr><td> Not reported</td><td>89</td><td>13.6</td><td>10</td><td>33.3</td></tr><tr><td colspan="5">Race</td></tr><tr><td> Asian</td><td>22</td><td>3.4</td><td>1</td><td>3.3</td></tr><tr><td> American Indian or Alaska Native</td><td>8</td><td>1.2</td><td>0</td><td>0</td></tr><tr><td> Black/African/African American</td><td>12</td><td>1.9</td><td>0</td><td>0</td></tr><tr><td> Caucasian/White</td><td>450</td><td>68.6</td><td>17</td><td>56.7</td></tr><tr><td> Native Hawaiian or Other Pacific Islander</td><td>2</td><td>0.3</td><td>0</td><td>0</td></tr><tr><td> Two or more</td><td>26</td><td>4.0</td><td>1</td><td>3.3</td></tr><tr><td> Other</td><td>24</td><td>3.7</td><td>1</td><td>3.3</td></tr><tr><td> Decline to answer/not reported</td><td>112</td><td>17.1</td><td>10</td><td>33.3</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note.</emph> EBP = evidence-based practices; COE = county office of education; SELPA = special education local plan area; DSP = designated specialist provider; SPED = special education; SLP = speech language pathologist; OT = occupational therapist.</p> <hd id="AN0162352852-8">Survey Sample</hd> <p>This study contains data for 656 survey respondents from the 28 SELPAs represented in the focus groups (described above; see also Table 1) who were primarily female (76.2%), White (68.6%), and special education teachers (33.9%), and had a master's degree (65.1%).</p> <hd id="AN0162352852-9">Procedures</hd> <p>This study was approved by the official institutional review board at UC Davis. Survey and focus group participants were informed that their data will not be linked to identifying information and will not be shared outside the research team. They were also informed that their supervisors will not have access to the information they provide. They were told that they do not have to answer any questions they do not wish to answer. Focus group data have not been reported elsewhere.</p> <hd id="AN0162352852-10">Focus Groups Procedures</hd> <p>CAPTAIN cadre received a recruitment email inviting them to participate. Focus groups took place in-person at the annual CAPTAIN cadre meeting in December 2017 (<reflink idref="bib5" id="ref38">5</reflink>) and via Zoom (<reflink idref="bib1" id="ref39">1</reflink>) in February 2018 for those who could not meet at the summit. Six 1.5-hr focus groups consisting of two to seven participants each were conducted and audio recorded for transcription.</p> <p>All facilitators were female, part of the research team, held a doctoral degree in psychology or special education, and had previous experience collecting qualitative data. Facilitators followed a structured interview guide with an overview of the session, primary questions, and additional guiding questions for each topic ([<reflink idref="bib10" id="ref40">10</reflink>]; [<reflink idref="bib21" id="ref41">21</reflink>]). Facilitators began by asking participants to think of a specific practice or initiative in their SELPA or district as the primary example for their responses. Participants discussed experiences in three areas: (a) practice selection (e.g., needs assessment and consideration of evidence base) and implementation (who received training; how was training provided); (b) administrator role in implementation (approvals needed, factors influencing approval or denial; resource allocation); and (c) training and coaching (factors related to effective training, trainee motivation and buy-in). Focus group questions were broad in order to facilitate discussion of individual experiences. Sample questions asked, "How are practices rolled out and maintained?" They were followed by probing questions such as "What factors affected the practice continuing or ending?" Participants received a US$20 gift card following participation.</p> <hd id="AN0162352852-11">Survey Procedures</hd> <p>As part of a larger study (see [<reflink idref="bib33" id="ref42">33</reflink>] for full study protocol), school personnel across California completed a survey asking about experiences with autism services, EBPs (knowledge, use, training), and implementation culture and climate. California educators and administrators were eligible to participate whether they worked in a public school setting that supported students with autism. The survey was available between May 2018 and March 2020. This study used demographic information, the Implementation Climate Scale (ICS), and Implementation Leadership Scale data (see below). This study uses a subset of those survey data completed by the 28 SELPAs (out of 133 in California) who had cadre representatives participating in the focus groups to facilitate triangulation of quantitative and qualitative data (which have not been reported elsewhere).</p> <hd id="AN0162352852-12">Implementation Climate Scale</hd> <p>This study used a combination of the original ICS ([<reflink idref="bib8" id="ref43">8</reflink>]) and a version adapted for schools (S-ICS; [<reflink idref="bib38" id="ref44">38</reflink>]) measuring perceptions of the policies, practices, procedures, and behaviors that are expected, rewarded, and supported to facilitate effective EBP implementation. Participants rated the extent to which they agreed with statements about autism EBP values and priorities from 0 (<emph>not at all</emph>) to 4 (<emph>very great extent</emph>). Participants completed six subscales from the S-ICS: (a) focus on EBPs, (b) educational support for EBPs, (c) recognition for EBPs (e.g., shout out at a staff meeting to teachers using an EBP successfully), (d) rewards for EBPs (i.e., monetary incentives to use EBPs such as bonuses or professional development funds), (e) use of data (to support EBPs), and (f) existing supports to deliver EBPs. They also completed two subscales from the ICS: selection for EBPs and selection for openness. The original ICS subscales demonstrate strong internal consistency (α =.81–.91; [<reflink idref="bib8" id="ref45">8</reflink>]). Subscale internal consistency was also strong in the school version (α =.85–.97, [<reflink idref="bib18" id="ref46">18</reflink>]).</p> <hd id="AN0162352852-13">School Implementation Leadership Scale</hd> <p>The School Implementation Leadership Scale (S-ILS; [<reflink idref="bib18" id="ref47">18</reflink>]) is a 30-item rating scale that assesses the degree to which a leader is (a) knowledgeable about EBPs, (b) supportive about the use of EBPs, (c) proactive about the use of EBPs, (d) perseverant in implementing EBPs, (e) communicates about EBPs, (f) has a vision for implementing EBPs, and (g) is available to discuss or implement EBPs. Participants rated the extent to which they agreed with statements from 0 (<emph>not at all</emph>) to 4 (<emph>very great extent</emph>). The S-ILS demonstrates excellent internal consistency reliability (α =.99) and convergent and discriminant validity. All subscales have an internal consistency above.94 ([<reflink idref="bib18" id="ref48">18</reflink>]). To reduce participant burden, the following subscales were included: Knowledgeable, Supportive, Proactive, and Perseverant. Direct service providers (e.g., classroom teachers and SLPs/OTs) selected and rated the primary leader for implementation of EBP for autism in their district (e.g., director of special education, principal, program manager/coordinator, and specialist). Nondirect service providers (e.g., specialists and program manager/coordinators) rated the leadership of their identified organizational level (e.g., district administration, COE administration, and SELPA administration). Participants in a leadership role (e.g., principals and directors) also rated their own leadership.</p> <hd id="AN0162352852-14">Data Analysis</hd> <p></p> <hd id="AN0162352852-15">Qualitative Data Analysis</hd> <p>NVivo QSR 11 ([<reflink idref="bib26" id="ref49">26</reflink>]) was used for qualitative analyses. A framework-driven analytic approach guided the coding process ([<reflink idref="bib12" id="ref50">12</reflink>]). In this approach, coders use framework constructs to develop codes and an iterative process to refine or add codes. Initial codes were based on the dimensions of the ICS ([<reflink idref="bib8" id="ref51">8</reflink>]; as adapted by [<reflink idref="bib38" id="ref52">38</reflink>]) (see measures section and definitions in Table 2) to support comparisons with quantitative data. Codes included the <emph>presence</emph> (e.g., my district provides EBP training twice a year) or <emph>absence</emph> (e.g., paraprofessionals do not get any EBP training in my district) of examples relating to each code. Any instance in which leadership was involved in the topic was coded to identify the general role of leadership in implementation. Additionally, the educational organization (SELPA, District, COE; School) was coded when applicable. The research team developed a codebook which contained operational definitions and examples of codes and detailed guidelines for use (available upon request). The coding scheme was applied to all focus group transcripts by two members of the research team who held either doctorate or master's degrees. Coders, consisting of members of the research team, independently coded each transcript and then met to resolve discrepancies and assign consensus codes. Codes that could not be resolved were brought back to the full team. Data were analyzed through NVivo coding queries to identify themes across dimensions of implementation climate and instances of leadership involvement. Matrix queries were conducted to identify patterns across SELPA, district, COE, and school levels. Themes are presented by dimensions of implementation climate. As part of the validation of findings, member checking by members of CAPTAIN was conducted. Findings were presented, and members were given the opportunity to provide feedback and reflect on the accuracy of the findings.</p> <p>Graph</p> <p>Table 2. Definitions and Integration of Qualitative and Quantitative Findings.</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /></colgroup><thead><tr><th align="left">Theme and definition</th><th align="center">Qualitative findings</th><th align="center">Quantitative findings</th><th align="center">Comparison</th></tr></thead><tbody><tr><td>Focus on EBP: Degree that use of the organization indicates quality EBP use is a priority and primary goal</td><td>Lack of focus on SPED and autism limits training; Focus needed at all levels to ensure success</td><td>District, School < SELPA, COE</td><td>SELPA, COE have better focus on EBP than districts, schools</td></tr><tr><td>Educational Support: How organizations support educational activities and training (providing EBP workshops, EBP materials, work time for training, etc.)</td><td>Provision of educational supports facilitates EBP use; SELPAs, COE provide more support than districts or schools</td><td><italic>District, School, COE < SELPA</italic><italic>School < COE, District</italic></td><td>Better support at the SELPA and COE levels across data</td></tr><tr><td>Recognition for EBPs: Ways providers are recognized for EBP expertise, use, and training. Can include level of esteem, recognition in meetings, nonmonetary awards, etc.</td><td>Limited mention, but increases buy-in and confidence; primarily at the coach level</td><td>District, School < SELPASchool < COE</td><td>Low reporting across data; if coaches work for SELPA and COE, data matches</td></tr><tr><td>Rewards for EBPs: Ways organizations reward EBP training and use using financial incentives, materials, bonuses, or raises.</td><td>Rarely mentioned; small rewards that can lead to promotion important; budget cuts have limited incentives</td><td>COE, District, School < SELPA</td><td>Low reporting across data; rarely used in schools</td></tr><tr><td>Selection for EBPs: How organizations take into account EBP knowledge and experience when hiring.</td><td>Not enough teachers available to select for EBPs</td><td>District, School < SELPA, COESchool < District</td><td>Low reporting across data; less selection at the school level</td></tr><tr><td>Selection for Openness: Flexibility, acceptability, and openness of providers; whether organizations base hiring decisions on these characteristics.</td><td>Barriers and facilitators based on teacher characteristics; strategies to increase openness.</td><td>District, School < SELPA</td><td>Discussed in relation to district, school; better ratings for SELPA</td></tr><tr><td>Use of Data: How organizations use data when implementing EBPs or to make decisions about training and implementation of EBPs</td><td>Rare at organizational level; often at coach–teacher level; examples at SELPA level</td><td>District, School < SELPA</td><td>Data used to seek approval at the SELPA level</td></tr><tr><td>Existing Supports for EBPs: Organizational strategies that support ongoing or future implementation of new practices generally</td><td>Discussed at the district and coach levels; barriers and facilitators to provide ongoing training</td><td>District, School < SELPA</td><td>Discussed at district level; better ratings for SELPA</td></tr></tbody></table> </ephtml> </p> <p>2 <emph>Note.</emph> EBPs = evidence-based practices; SELPA = special education local plan area; COE = county office of education; SPED = special education.</p> <hd id="AN0162352852-16">Quantitative Data Analysis</hd> <p>Quantitative analyses were performed using SPSS V.26. Bonferroni-corrected paired <emph>t</emph> tests were used to compare within-subject responses on the ICS and ILS subscales. Bonferroni-corrected analysis of variance (ANOVA) was used to compare ICS and ILS responses across administrative and response levels. Bonferroni-corrected post hoc comparisons were conducted when the omnibus <emph>F</emph>-test was significant. At the individual participant level, any participants with data on any of the primary measures were included and any subscales with complete data were included.</p> <hd id="AN0162352852-17">Results</hd> <p>Themes are presented across implementation climate and leadership domains.</p> <hd id="AN0162352852-18">Implementation Climate: Qualitative Results</hd> <p></p> <hd id="AN0162352852-19">Focus on EBPs</hd> <p>Participants reported both a presence and an absence of focus on EBPs in their organizations at different levels as implementation determinants. Participants indicated prioritization and support from administration at the SELPA, district, and school level as well as by teachers and staff at the school level was key to support EBP implementation:</p> <p>That's what I have learned now is that you really need commitment from the leadership saying, yes this is a priority, yes we are going to move this forward, we are going to make evidence-based practices and implementing them a priority and once that's done then you're going to have the support that you need.... We also went ahead and somebody from the diagnostic center came out and talked to all the principals because we have leadership meetings and so everybody is aware of the program. So, it's a district wide program, everybody's aware so they know that down the road it's going to be an expectation that if you have an autism specific classroom, your teachers are going to have to be eventually at some part of this rollout.</p> <p>Participants report a lack of prioritization of special education and autism programs as a significant barrier to EBP use as evidenced by limited time for professional development and a lack of available substitute teachers to allow teachers to attend training:</p> <p>I think one of our big struggles is professional development for our Special Ed teachers. General Ed teachers get a ton of professional development. And so, if we ask for Special Ed time, there are only so many subs that we are allowed per day in our district, and so all those subs are already allocated. Then we get a slot of 2 days the first week of school, which is not really appropriate. Of all people, our Special Ed teachers need to be in class the first couple weeks getting the routines established.</p> <p>Participants also discussed how school climate and culture impacts the prioritization of programs and practices:</p> <p>So, I find you know it's belief systems that come from the family or within the culture of the school itself. So, starting at the top at their admin level, with the principal. What are the beliefs, the majority belief, within that system that they're in. And what support do they have from that system to do it differently?</p> <hd id="AN0162352852-20">Educational Support for EBPs</hd> <p>Participants generally reported a presence of educational support for EBPs. At the school level, participants reported using a myriad of strategies including hosting trainings with "make-and-takes" and creating accessible, online resources. Participants reported supporting teaching staff by involving teachers in the EBP decision-making process to gain buy-in for implementation of new practices:</p> <p>Well, we have now what we call the "classroom re-assurance indicators" and how all that started is a team was developed... and so we kind of sat down and talked about where we wanted to see changes in our classrooms, and what kind of supports we needed to provide to our teachers.</p> <p>At the district level, participants reported involving outside consultants for specialized trainings and creating district-wide professional learning communities. They also discussed barriers to district-level educational supports such as challenges obtaining funding approval and union restrictions on training time:</p> <p>We started two years ago, a CAPTAIN Coaches Network, because we're so big and so spread out. So, what we did is we partnered with [consultant] and we got this whole project going where she came in and she does the trainings. We do some EBP trainings, some coaching trainings, we coach coaches on how to be coaches and then teach them about EBPs.</p> <p>Participants generally reported more support for roll out of educational supports from the SELPA and COE than at the district or school level:</p> <p>We have a [SELPA] director who fully supports [EBPs] and is actually pushing it and is not just like "yeah, whatever you need," but really on us about it as well. And he's given us laminators for every school site, colored printers for every site. He gave us the funds that we needed to make the things [the teachers] really need because that was where we kept backlogging; we didn't have the materials and the resources to even do the evidence-based practices we wanted to do and then release time and paid professional development time.</p> <hd id="AN0162352852-21">Recognition for EBPs</hd> <p>Participants discussed recognition less frequently than other domains. However, participants noted some examples of how they are able to individually recognize teachers. They described hosting "showcases" where teachers can show off their work and sending recognition emails. They reported positive benefits of recognition of teacher efforts:</p> <p>We asked both [teams] and pulled on [the teacher's] strength and believe it or not, after she gave her presentation, when we went in to coach the next time, there was a lot more motivation on her part just having recognized [data collection] for her. It really boosted her and made her I guess feel like there were some things she was doing well.</p> <p>Participants discussed how sharing what teachers are doing with the coach and other teachers is a form of recognition that increases buy-in for training and coaching:</p> <p>I think that has buy-in for the teachers especially who really do want to do better, because then it's like "Oh, I get to show you what I'm doing?" They want to show off what they are doing. Or I'll even say if you can show us or bring a video model of what you do [to the job-alikes].</p> <hd id="AN0162352852-22">Rewards for EBPs</hd> <p>Participants rarely discussed rewards for EBPs and instead noted the impact of budget cuts or union restrictions at the district level on their ability to use incentives for participation in professional development or training:</p> <p>Part of our initial budget was to be able provide stipends for people. And then we couldn't do it after that because they said wait a minute, we can't do that because of union stuff or whatever.</p> <p>However, participants noted how they use rewards such as snacks, continuing education hours and materials to incentivize and engage teachers in EBP training and coaching sessions:</p> <p>We give them continuing [education] hours for doing [the trainings] and we've seen an increase since we did that because they are getting something out of it.</p> <p>So, we've rolled out individual trainings on all these different things and a lot of those things have salary point classes for our teachers to attend, which move them forward on their pay scale.</p> <hd id="AN0162352852-23">Selection for EBPs</hd> <p>Participants rarely discussed selection but did mention that teachers are often hired without any EBP experience because there is such a shortage of teachers:</p> <p>The lack of teachers, I mean we're having teachers that are being hired that don't have credentials, and they're like emergency credentials, and they don't have any experience and they don't even know how to teach, let alone, they don't know the basis of being able to run a classroom. So, going in and talking to them about evidence-based practices, like they don't even know enough to get to that point yet.</p> <p>Participants noted that because so many teachers are new, it is not a priority to receive training on EBP because they needed to focus on required training:</p> <p>One of our main difficulties throughout the county is that a lot of our staff are new, and they rotate, and so we have a lot of brand-new teachers. So, whenever you're trying coach them on EBPs, they're still not sure of how to fill out IEP paperwork, they're not sure on how to implement the curriculum and so we had to like take a step back and figure out how do we train these teachers to be teachers and then how do we train them to implement EBPs.</p> <hd id="AN0162352852-24">Selection for Openness</hd> <p>Participants less frequently discussed selection for openness at the organizational level, but did report several factors that affect a teacher's openness to learn about and implement EBP. Barriers to EBP implementation included experienced teachers who did not feel the need to change their strategies and teachers who were dissatisfied or overwhelmed:</p> <p>Our veteran teachers, the ones we kind of force to go, aren't usually happy with us. They don't feel that they need to, that what they are doing is just fine.</p> <p>And so, they're so upset and frustrated with their own difficulties within their district and at their school sites that they're not open or willing to learn. And so that makes it really difficult whenever I have done some of the district coaching and training pieces.</p> <p>Participants noted that teachers who are newer to the profession and "excited" or those who have some EBP experience are more open to training and coaching:</p> <p>People who are already interested in autism, people who already have some baseline and kind of building capacity that way like you are already on board.</p> <p>Lastly, participants discussed strategies they use to build rapport with teachers to facilitate training and coaching, including setting clear expectations, directly helping teachers in the classroom, and emphasizing that EBPs can be used for all students:</p> <p>I come in and help you work with that student right where they are. Let's try doing some of these EBP because they do really work. But what I'll do then is start modeling it first so that way they'll see that I'm not expecting perfection. And so, from there, it opens them up to saying ok "What did you do and how did you do that?." "Okay, let's sit down now and we can plan and talk and do," you know?</p> <hd id="AN0162352852-25">Use of Data</hd> <p>Only a few participants discussed use of data by their organizations for EBP decision-making. Those that did mention it highlighted the critical role that data collection plays in obtaining approval from leadership for trainings and resources:</p> <p>My SELPA director is very data oriented and if you can show her the data, then [she is] going to have a much better idea of things and it kind of helps. It's like "I know your perception is that we're stellar, but look these areas, the data shows that we have these weaknesses."</p> <p>Participants noted using a variety of data collection methods during training and coaching including goal attainment scaling, fidelity checklists, and self-evaluations, all of which foster engagement in teachers by allowing them to track their own progress:</p> <p>We put together a goal attainment scale on the things we want to see from them, and one is meeting fidelity and then they have their student's goals and they're monitoring that, so they're seeing as their progress is growing, so is their student's, and so that is always really exciting.</p> <hd id="AN0162352852-26">Existing Supports for EBPs</hd> <p>Participants discussed existing supports most often in relation to district support. Participants discussed barriers that impede them from providing ongoing supports including a lack of authority in ensuring training participation:</p> <p>And I feel very frustrated because we are not getting anything from up on the top, I have no authority whatsoever to say sorry you got to, you have to make time for this. And I feel that is the only way, what I need to be able to do this.</p> <p>Participants reported strategies they individually used to provide ongoing support to teachers including follow-up coaching, modeling, and feedback:</p> <p>We go into classrooms and we coach, and model and we follow-up with [the teachers] on trainings for these things. We're actually going in and saying look, here's this, this is how to do it, let me show you how to do it, let's do it together. We go through the whole coaching process.</p> <p>Interestingly, participants discussed how litigation and due process helped facilitate EBP training by hiring experienced staff to train teachers:</p> <p>Our districts have also looked at and hired their own autism or behavior coordinator as well. And then it's also brought a lot more BCBAs to the area, and to the districts, and then to the county office. And what supported it was just significant reduction in the amount of lawsuits, the amount of due process that we were going through and then also, significant increase in our ability to retain personnel, specifically our teachers because then they were getting the training that they needed on EBP because they were being trained by knowledgeable up-to-date people.</p> <hd id="AN0162352852-27">Implementation Climate: Quantitative Results</hd> <p></p> <hd id="AN0162352852-28">Implementation Climate Scale</hd> <p>As shown in Table 3, across all survey respondents, low levels of all subscales were reported. Of all the subscales, Focus (<emph>M</emph> = 2.04, <emph>SD</emph> = 1.18) and Selection for Openness (<emph>M</emph> = 2.36, <emph>SD</emph> = 1.11) had the highest ratings. Selection for Openness was rated significantly higher than all other ICS domains (<emph>M</emph> = 2.36, <emph>SD</emph> = 1.11, <emph>t</emph> > 9.3, <emph>p</emph> <.001). Rewards for EBPs was rated significantly lower than all other ICS domains (<emph>M</emph> = 0.86, <emph>SD</emph> = 1.03, <emph>t</emph> > 13.8, <emph>p</emph> <.001).</p> <p>Graph</p> <p>Table 3. Implementation Climate and Implementation Leadership Means (Quantitative).</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /></colgroup><thead><tr><th align="left">Measure</th><th align="center"><italic>n</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th></tr></thead><tbody><tr><td colspan="4">Implementation climate</td></tr><tr><td> Focus on EBPs</td><td>582</td><td>2.04</td><td>1.18</td></tr><tr><td> Educational Support for EBPs</td><td>576</td><td>1.84</td><td>1.24</td></tr><tr><td> Recognition for EBPs</td><td>544</td><td>1.88</td><td>1.12</td></tr><tr><td> Rewards for EBPs</td><td>549</td><td>0.86</td><td>1.03</td></tr><tr><td> Selection for EBPs</td><td>530</td><td>1.68</td><td>1.17</td></tr><tr><td> Selection for Openness</td><td>555</td><td>2.36</td><td>1.11</td></tr><tr><td> Use of Data</td><td>540</td><td>1.48</td><td>1.09</td></tr><tr><td> Existing Supports for EBPs</td><td>547</td><td>1.54</td><td>1.20</td></tr><tr><td colspan="4">Implementation leadership</td></tr><tr><td> Proactive</td><td>531</td><td>2.09</td><td>1.15</td></tr><tr><td> Knowledgeable</td><td>519</td><td>2.50</td><td>1.21</td></tr><tr><td> Supportive</td><td>537</td><td>2.50</td><td>1.15</td></tr><tr><td> Perseverant</td><td>514</td><td>2.39</td><td>1.13</td></tr><tr><td colspan="4">Implementation leadership (Self)</td></tr><tr><td> Proactive</td><td>354</td><td>2.26</td><td>1.06</td></tr><tr><td> Knowledgeable</td><td>362</td><td>2.68</td><td>0.99</td></tr><tr><td> Supportive</td><td>350</td><td>3.06</td><td>0.80</td></tr><tr><td> Perseverant</td><td>347</td><td>2.87</td><td>0.81</td></tr></tbody></table> </ephtml> </p> <p>3 <emph>Note</emph>. Scores range from 0 (<emph>not at all</emph>) to 4 (<emph>very great extent</emph>) for all scales. EBPs = evidence-based practices.</p> <hd id="AN0162352852-29">Implementation Climate by Program Level</hd> <p>As seen in Table 4, SELPAs had significantly better implementation climate across all domains compared with districts and schools (<emph>p</emph> <.001) and greater use of Educational Supports, Existing Supports, and Rewards than COEs (<emph>p</emph> <.02). The COEs had higher Focus and Selection for EBP than districts and schools, and better Educational Supports and Recognition than schools (<emph>p</emph> <.04). Districts had higher Educational Supports, Selection for EBP, and Existing Supports for EBP than schools (<emph>p</emph> <.03).</p> <p>Graph</p> <p>Table 4. Implementation Climate by Program Level (Quantitative).</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /></colgroup><thead><tr><th align="left" rowspan="2">ICS subscale</th><th align="center" colspan="2">School</th><th align="center" colspan="2">District</th><th align="center" colspan="2">COE</th><th align="center" colspan="2">SELPA</th><th align="center" rowspan="2"><italic>F</italic></th><th align="center" rowspan="2">Post hoc<xref ref-type="table-fn" rid="tfn5">a</xref></th></tr><tr><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th></tr></thead><tbody><tr><td>Focus</td><td>1.89</td><td>1.18</td><td>1.89</td><td>1.13</td><td>2.59</td><td>0.82</td><td>2.91</td><td>0.97</td><td>18.52</td><td>D, SC < SE, C</td></tr><tr><td>Educational support</td><td>1.49</td><td>1.18</td><td>1.91</td><td>1.11</td><td>2.29</td><td>0.99</td><td>3.12</td><td>0.91</td><td>40.68</td><td>D, SC, C < SE, SC < C, D</td></tr><tr><td>Recognition</td><td>1.63</td><td>1.08</td><td>1.88</td><td>1.08</td><td>2.37</td><td>0.67</td><td>2.83</td><td>0.94</td><td>24.35</td><td>D, SC < SE, SC < C</td></tr><tr><td>Rewards</td><td>0.79</td><td>1.05</td><td>0.81</td><td>0.93</td><td>0.60</td><td>0.60</td><td>1.42</td><td>1.11</td><td>7.55</td><td>C, D, SC < SE</td></tr><tr><td>Selection for EBP</td><td>1.37</td><td>1.11</td><td>1.74</td><td>1.09</td><td>2.32</td><td>1.09</td><td>2.67</td><td>0.94</td><td>28.47</td><td>D, SC < SE, C, SC < D</td></tr><tr><td>Selection for Openness</td><td>2.15</td><td>1.16</td><td>2.37</td><td>1.01</td><td>2.54</td><td>0.94</td><td>3.13</td><td>0.86</td><td>15.98</td><td>D, SC < SE</td></tr><tr><td>Use of Data</td><td>1.31</td><td>1.10</td><td>1.41</td><td>1.03</td><td>1.66</td><td>0.88</td><td>2.25</td><td>0.97</td><td>15.14</td><td>D, SC < SE</td></tr><tr><td>Existing Supports for EBP</td><td>1.22</td><td>1.12</td><td>1.54</td><td>1.11</td><td>1.79</td><td>1.06</td><td>2.78</td><td>0.94</td><td>38.24</td><td>C, D, SC < SE, SC < D</td></tr></tbody></table> </ephtml> </p> <ulist> <item>4 <emph>Note.</emph> Scores range from 0 (<emph>not at all</emph>) to 4 (<emph>very great extent</emph>). ICS = Implementation Climate Scale; COE = county office of education; SELPA = special education local plan area; EBPs = evidence-based practices; C = COE; D = District; SC = School; SE = SELPA.</item> <item>5 Post hoc analysis with a Bonferroni-corrected <emph>p</emph> <.05 is reported.</item> </ulist> <hd id="AN0162352852-30">Leadership: Qualitative Results</hd> <p>Participants discussed leadership frequently in various contexts and across several of the implementation climate domains. Presence of leadership was most frequently discussed at the COE and SELPA levels. Leadership support varied across organizations, for example increased support at the SELPA level, and pushback at the district level:</p> <p>I would say our champion was definitely our SELPA director who had relationship and trust with everybody so he could have those closed-door conversations [with other administrators] about, you know, "Would you support this?" So, before we even bring it to our governance for vote we know typically what's going to happen, who's going to vote which way.</p> <p>Participants discussed how principals play a key role in supporting teacher training:</p> <p>As far as getting teachers to be able to attend the trainings if its during class time, that is still dependent on the onsite administrator.</p> <p>Other important leadership factors included maintaining open communication and building a trusting relationship with leadership, and the role of leader buy-in for prioritizing a focus on special education training, and program success:</p> <p>I felt very much like, I could feel [the director's] buy-in. And he only had that because the district he was at previously had an autism team, and he saw the effects of it. So, I didn't have to create the buy-in, the buy-in was already there. He's like "I love this, let's do it, I want to meet with the CAPTAIN team." Full support.</p> <p>Conversely, a lack of leadership limited available resources and reduced teacher buy-in for EBP implementation:</p> <p>[Our director] was supportive, but not financially on time or release. It was like as long as this doesn't cost money or make any waves, like go for it. Which it worked a little, but we didn't see a huge change.</p> <hd id="AN0162352852-31">Leadership: Quantitative Results</hd> <p></p> <hd id="AN0162352852-32">Implementation Leadership Scale</hd> <p>As seen in Table 3, across all survey respondents, moderate levels of all subscales were reported. Knowledgeable (<emph>M</emph> = 2.50, <emph>SD</emph> = 1.21, <emph>t</emph> > 3.17, <emph>p</emph> <.002) and Supportive leadership (<emph>M</emph> = 2.50, <emph>SD</emph> = 1.15, <emph>t</emph> > 5.23, <emph>p</emph> <.001) were rated significantly higher than Proactive and Perseverant leadership. Proactive leadership was rated significantly lower than all other ILS domains (<emph>M</emph> = 2.09, <emph>SD</emph> = 1.15, <emph>t</emph> > 8.04, <emph>p</emph> <.001).</p> <hd id="AN0162352852-33">Implementation Leadership Self-ratings</hd> <p>As seen in Table 5, participants that self-identified as a leader (e.g., supervisors and trainers) in a SELPA rated themselves as having higher Proactive, Knowledgeable, Supportive, and overall implementation leadership than leaders at the district level rated themselves (<emph>p</emph> <.04). The COE leaders had higher self-rated Knowledgeable implementation leadership than district leaders (<emph>p</emph> <.01). The SELPA and COE leaders also had higher self-rated Knowledgeable, Supportive, Perseverant, and average implementation leadership than school administrators (<emph>p</emph> <.04).</p> <p>Graph</p> <p>Table 5. Implementation Leadership Self-Ratings by Leaders (Quantitative).</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /></colgroup><thead><tr><th align="left" rowspan="2">ILS subscale</th><th align="center" colspan="2">School</th><th align="center" colspan="2">District</th><th align="center" colspan="2">COE</th><th align="center" colspan="2">SELPA</th><th align="center" rowspan="2"><italic>F</italic></th><th align="center" rowspan="2">Post hoc<xref ref-type="table-fn" rid="tfn7">a</xref></th></tr><tr><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th></tr></thead><tbody><tr><td>Proactive</td><td>2.20</td><td>1.16</td><td>2.15</td><td>1.03</td><td>2.27</td><td>1.10</td><td>2.61</td><td>0.82</td><td>3.02</td><td>D < SE</td></tr><tr><td>Knowledgeable</td><td>2.35</td><td>1.05</td><td>2.70</td><td>0.94</td><td>3.40</td><td>0.70</td><td>3.10</td><td>0.78</td><td>13.84</td><td>D, SC < SE, C</td></tr><tr><td>Supportive</td><td>2.85</td><td>0.90</td><td>3.04</td><td>0.76</td><td>3.43</td><td>0.50</td><td>3.46</td><td>0.53</td><td>10.00</td><td>D, SC < SE, SC < C</td></tr><tr><td>Perseverant</td><td>2.71</td><td>0.93</td><td>2.86</td><td>0.75</td><td>3.24</td><td>0.67</td><td>3.11</td><td>0.63</td><td>4.93</td><td>SC < SE, C</td></tr><tr><td>ILS-Self Average</td><td>2.54</td><td>0.91</td><td>2.71</td><td>0.72</td><td>3.13</td><td>0.57</td><td>3.07</td><td>0.53</td><td>8.19</td><td>D, SC < SE, SC < C</td></tr></tbody></table> </ephtml> </p> <ulist> <item>6 <emph>Note.</emph> Scores range from 0 (<emph>not at all</emph>) to 4 (<emph>very great extent</emph>). ILS = Implementation Leadership Scale; COE = county office of education, SELPA = special education local plan area; C = COE; D = District; SC = School; SE = SELPA.</item> <item>7 Post hoc analysis with a Bonferroni-corrected <emph>p</emph> <.05 is reported.</item> </ulist> <hd id="AN0162352852-34">Implementation Leadership by Respondent Level</hd> <p>As seen in Tables 6 and 7, SELPA directors were rated by specialists and other administrators as having significantly higher implementation leadership across all domains than district administrators (<emph>p</emph> <.0001), and significantly higher Proactive, Supportive, Perseverant, and average implementation leadership than school administrators (<emph>p</emph> <.02). The COE administrators were rated as having higher Knowledgeable implementation leadership than district administrators (<emph>p</emph> <.001).</p> <p>Graph</p> <p>Table 6. Implementation Leadership Ratings of Non-DSP (Quantitative).</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /></colgroup><thead><tr><th align="left" rowspan="3">ILS subscale</th><th align="center" colspan="8">Non-DSP ratings</th><th align="center" rowspan="3"><italic>F</italic></th><th align="center" rowspan="3">Post hoc<xref ref-type="table-fn" rid="tfn9">a</xref></th></tr><tr><th align="center" colspan="2">School admin</th><th align="center" colspan="2">District admin</th><th align="center" colspan="2">COE admin</th><th align="center" colspan="2">SELPA director</th></tr><tr><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th></tr></thead><tbody><tr><td>Proactive</td><td>1.84</td><td>1.00</td><td>1.73</td><td>1.09</td><td>2.10</td><td>0.81</td><td>2.56</td><td>1.14</td><td>6.80</td><td>D, SC < SE</td></tr><tr><td>Knowledgeable</td><td>2.08</td><td>1.16</td><td>1.73</td><td>1.06</td><td>2.69</td><td>0.93</td><td>2.62</td><td>1.12</td><td>9.81</td><td>D < SE, C</td></tr><tr><td>Supportive</td><td>2.06</td><td>0.89</td><td>2.13</td><td>1.08</td><td>2.68</td><td>1.07</td><td>3.15</td><td>0.91</td><td>12.35</td><td>D, SC < SE</td></tr><tr><td>Perseverant</td><td>2.00</td><td>0.97</td><td>1.90</td><td>0.96</td><td>2.41</td><td>0.78</td><td>2.89</td><td>1.05</td><td>11.39</td><td>D, SC < SE</td></tr><tr><td>ILS Average</td><td>1.98</td><td>0.93</td><td>1.89</td><td>0.98</td><td>2.44</td><td>0.68</td><td>2.85</td><td>0.93</td><td>11.17</td><td>D, SC < SE</td></tr></tbody></table> </ephtml> </p> <ulist> <item>8 <emph>Note.</emph> Scores range from 0 (<emph>not at all</emph>) to 4 (<emph>very great extent</emph>). DSP = designated specialist provider; ILS = Implementation Leadership Scale; admin = administration; COE = county office of education; SELPA = special education local plan area; C = COE; D = District; SC = School; SE = SELPA.</item> <item>9 Post hoc analysis with a Bonferroni-corrected <emph>p</emph> <.05 is reported.</item> </ulist> <p>Graph</p> <p>Table 7. Implementation Leadership Ratings by Respondent Type (Quantitative).</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /></colgroup><thead><tr><th align="left" rowspan="3">ILS subscale</th><th align="center" colspan="10">DSP ratings</th><th align="center" rowspan="3"><italic>F</italic></th><th align="center" rowspan="3">Post hoc<xref ref-type="table-fn" rid="tfn11">a</xref></th></tr><tr><th align="center" colspan="2">SPED director</th><th align="center" colspan="2">Principal</th><th align="center" colspan="2">Program manager</th><th align="center" colspan="2">Specialist</th><th align="center" colspan="2">Teacher</th></tr><tr><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th><th align="center"><italic>M</italic></th><th align="center"><italic>SD</italic></th></tr></thead><tbody><tr><td>Proactive</td><td>1.57</td><td>1.21</td><td>2.41</td><td>1.29</td><td>2.24</td><td>1.17</td><td>2.20</td><td>1.09</td><td>2.52</td><td>1.21</td><td>4.97</td><td>DS < S, T</td></tr><tr><td>Knowledgeable</td><td>2.04</td><td>1.24</td><td>2.67</td><td>1.56</td><td>3.01</td><td>0.93</td><td>3.06</td><td>1.07</td><td>2.74</td><td>1.12</td><td>7.20</td><td>DS < PM, S, T</td></tr><tr><td>Supportive</td><td>2.03</td><td>1.29</td><td>2.61</td><td>1.38</td><td>2.66</td><td>1.12</td><td>2.72</td><td>1.07</td><td>2.69</td><td>1.19</td><td>3.87</td><td>DS < S, T</td></tr><tr><td>Perseverant</td><td>1.77</td><td>1.29</td><td>2.53</td><td>1.44</td><td>2.64</td><td>1.05</td><td>2.65</td><td>1.07</td><td>2.82</td><td>1.11</td><td>6.43</td><td>DS < PM, S, T</td></tr><tr><td>ILS Average</td><td>1.83</td><td>1.25</td><td>2.43</td><td>1.39</td><td>2.68</td><td>0.98</td><td>2.68</td><td>0.98</td><td>2.60</td><td>1.08</td><td>4.86</td><td>DS < PM, S, T</td></tr></tbody></table> </ephtml> </p> <ulist> <item>10 <emph>Note.</emph> Scores range from 0 (<emph>not at all</emph>) to 4 (<emph>very great extent</emph>). DSP = designated specialist provider; ILS = Implementation Leadership Scale; DS = director of special education, PM = program manager; S = specialist; T = teacher; SPED = special education.</item> <item>11 Post hoc analysis with a Bonferroni-corrected <emph>p</emph> <.05 is reported.</item> </ulist> <p>As noted, direct service providers (e.g., paraprofessionals, site teachers, and SLPs/OTs) first selected the role that provides primary leadership for implementation of EBP for autism in their district and then provided ratings on the implementation leadership of the specific person in that role. As seen in Tables 6 and 7, direct service providers rated directors of special education as having significantly lower implementation leadership across all domains compared with the implementation leadership of program/autism/behavior specialists and teachers (<emph>p</emph> <.02) and lower Knowledgeable, Perseverant, and average implementation leadership compared with program managers/coordinators.</p> <hd id="AN0162352852-35">Discussion</hd> <p>This mixed methods study explored implementation climate and leadership as a step toward identification of perceived variables linked to effective statewide scale-up of EBPs for autism within special education. In general, the mixed methods analyses indicated convergence and triangulation of themes, with some unique findings emerging from each approach.</p> <hd id="AN0162352852-36">Implementation Climate</hd> <p>Data indicate clear convergence across several domains of implementation climate between cadre trainers and other leaders, teachers, and direct service providers in their SELPA regions (see Table 2). For example, Selection for Openness to EBPs was indicated as a strength across organizational levels in both cadre focus groups (although not discussed often) and broader surveys, whereas Reward for EBP use was rated as low on quantitative measures and mentioned minimally during focus groups. Similarly, previous studies of school-based mental health providers found Focus and Selection for Openness rated the highest and Rewards rated the lowest ([<reflink idref="bib18" id="ref53">18</reflink>]).</p> <p>Selection for openness may be a common practice in special education. The lack of standardization in curriculum and teaching practices used across districts or in teacher preparation programs means that education agencies might prefer candidates open to learning the practices adopted within their district. Rewarding EBP use, however, can be difficult within educational contexts in part due to limited budgets for this type of expenditure and language within collective bargaining agreements that prohibit additional compensation. Despite the reported barriers to reward, qualitative evidence suggests methods to overcome these barriers (i.e., salary point opportunities). Yet, quantitative data suggest that efforts to implement EBP were not rewarded often at the school or district level.</p> <p>While Focus on EBPs had the second highest mean, compared with previous literature ([<reflink idref="bib18" id="ref54">18</reflink>]) averages were low and participants reported barriers during the focus groups. This is not entirely surprising within special education since there has historically been a much higher emphasis on Individuals with Disabilities Education Act (IDEA) compliance than on student outcomes. A shift began in 2014 with the passage of result-driven accountability ([<reflink idref="bib39" id="ref55">39</reflink>]), which requires states and districts to outline strategic methods for improving outcomes, inclusive of research-based practices. This policy shift may increase focus on EBPs as systems mobilize to meet this mandate.</p> <p>While our data indicate similar implementation climate ratings for COEs and SELPAs, we see much lower climate ratings for schools and districts with significant differences across all domains. Although there was some variability, overall participants rated SELPAs as having higher implementation climate than schools, districts, or COEs. Qualitative data demonstrated a similar thematic pattern of more positive ratings related to implementation at the SELPA level.</p> <p>The comparatively higher implementation climate scores seen in SELPAs may be a result of their highly specific focus on students with disabilities, whereas districts and schools have a much broader focus and mandate to support all students. There may also be a difference in the background knowledge about autism and EBPs held by SELPA staff and leaders versus those at the district and school site levels, as illustrated in the data regarding leadership. Together, these outcomes indicate a clear need for interventions related to improving implementation climate for educational organizations and providers supporting students with autism. The prioritization of general education is evident in qualitative and quantitative findings. Indeed, participants reported that general education teachers receive more consistent professional development than special education teachers. Findings related to the lack of special education-focused professional development were confirmed by quantitative data.</p> <hd id="AN0162352852-37">Implementation Leadership</hd> <p>Overall, implementation leadership was rated in the low to moderate range, suggesting this as an area for potential growth. This is consistent with existing literature indicating challenges in implementation leadership in education settings and is not surprising given the limited training most administrators receive in system-level change using research-based implementation supports ([<reflink idref="bib17" id="ref56">17</reflink>]; [<reflink idref="bib20" id="ref57">20</reflink>]; [<reflink idref="bib23" id="ref58">23</reflink>]). Previous studies in schools have reported subscale averages ranging between 1.78 and 2.4 ([<reflink idref="bib18" id="ref59">18</reflink>]), with Supportive leadership having the highest mean (<emph>M</emph> = 2.4) and Proactive leadership having the lowest mean (<emph>M</emph> = 1.78). Similarly, the current sample rated Supportive leadership highest on average, across respondent types (self, other) and organization types and Proactive leadership was consistently rated lower than the other domains. These outcomes indicate special education leaders may need targeted training and support in how to guide exploration activities to support proactive implementation of EBPs for autism. Similarly, to climate, implementation leadership was rated highest at the SELPA and COE level perhaps due to the specialized focus on special education.</p> <p>Across all organization types and domains, leaders' self-ratings of implementation leadership were higher than respondent ratings, suggesting differences in perceived behavior. Organizational leaders may engage in implementation leadership activities that others may not be aware of, and, therefore, are rated lower by other nonleader staff. It is also possible that leader intentions to support EBP implementation may not impact other organizational staff in a meaningful way. For this study, qualitative data were only collected from midlevel staff, not organizational leaders; this is an area that warrants additional examination in future research.</p> <hd id="AN0162352852-38">Implications for Practice</hd> <p>Given the low-to-moderate perceptions of implementation climate and leadership as reported by the participants in our study, and existing literature that supports the importance of these factors in EBP implementation and sustainment in schools ([<reflink idref="bib15" id="ref60">15</reflink>]; [<reflink idref="bib20" id="ref61">20</reflink>]; [<reflink idref="bib22" id="ref62">22</reflink>]), targeted leadership training is needed. For example, Leadership and Organizational Change for Implementation (LOCI) is an organizational implementation strategy that targets implementation leadership to improve the organizational context to support EBP implementation ([<reflink idref="bib2" id="ref63">2</reflink>]). LOCI has been tested in health service settings and is currently being tested in school settings specifically for implementation of EBPs for autism ([<reflink idref="bib6" id="ref64">6</reflink>]). Our findings indicated differences in perceptions of implementation leadership across organizational levels, which have implications for building capacity at the district and school levels. Since SELPAs and COEs are often in charge of providing training and technical assistance to district and school sites, perhaps SELPA and COE leaders could assist school and district leaders in building implementation climate and leadership. Furthermore, while the SELPA structures are unique to California, most states have regional entities that help deliver special education services, meaning these findings can have implications nationwide.</p> <p>Our findings also indicated that implementation climate was stronger at the SELPA and COE level and the qualitative data indicated that this may be due to stronger focus on supporting the greater student population (e.g., general education) in districts and schools. In fact, the linguistic cues within the qualitative data highlight perceptions of othering between the two groups (general education vs. special education) at the school level. This can impact school climate, as well as the providers' perceptions of openness and buy-in from school-level administrators. Given the increasing number of children and adolescents with autism educated in general education settings, there is a need to bridge the divide between general and special education. Targeted intervention focused on incorporating professional development related to EBPs for autism for all teachers may support greater inclusion of autistic students and coordination between special and general education settings. If general education is a school-level priority, then entering EBPs for autism into the general education professional development docket may be an avenue to promote more buy-in from school-level administrators. This may require linguistic reframing from a general education versus special education narrative to teaching all children under the umbrella of universal design or differentiated instruction.</p> <hd id="AN0162352852-39">Limitations and Implications for Future Research</hd> <p>Several limitations should be noted. First, our sample was geographically located in one state, California. Future studies should include representative samples from regions across the United States. Second, data were collected during one time period (1 school year). Future studies should collect data across multiple time periods to examine changes in implementation climate and leadership. Third, it is possible the structure of the focus guide questions may have limited discussion of certain domains of implementation climate. Future studies could specifically include questions and examples regarding different aspects of implementation climate. Fourth, due to the nature of the study, we were unable to include many validation strategies for the qualitative data. Future studies should ensure the validity of results by employing strategies such as audit trails and peer debriefing. Fifth, there is a possibility that the authors' collaborative work with CAPTAIN could result in bias. Future studies should employ exercises related to reflexivity to diminish the possibility of bias. Sixth, our quantitative data came from smaller participant groups at some organizational levels (e.g., COE). Future studies might ensure comparable and representative samples by recruiting at each organizational level. Seventh, while the findings provide insight on perceptions of implementation climate and leadership in schools, we cannot confirm how they predict implementation of EBPs. Future studies should use predictive statistical models to analyze the relationship between these variables. Lastly, this study did not collect data on student outcomes. Future studies could compare student outcomes across schools with poor and strong implementation climate and leadership in order to examine their association.</p> <hd id="AN0162352852-40">Conclusion</hd> <p>In sum, the findings from this study suggest implementation climate and leadership for EBPs are rated moderate to low in special education. However, variability between organization levels was detected, with more specialized organizations like SELPAs and COEs rating higher for both implementation climate and leadership. Further research is warranted to continue exploring the influence of organizational mechanisms on the implementation of EBPs for autism in schools.</p> <ref id="AN0162352852-41"> <title> References </title> <blist> <bibl id="bib1" idref="ref19" type="bt">1</bibl> <bibtext> Aarons G. A., Ehrhart M. G., Farahnak L. R. (2014). The Implementation Leadership Scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9(1), Article 45. https://doi.org/10.1186/1748-5908-9-45</bibtext> </blist> <blist> <bibl id="bib2" idref="ref63" type="bt">2</bibl> <bibtext> Aarons G. A., Ehrhart M. G., Farahnak L. R., Hurlburt M. S. (2015). Leadership and organizational change for implementation (LOCI): A randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10(1), 1–12. https://doi.org/10.1186/s13012-014-0192-y</bibtext> </blist> <blist> <bibl id="bib3" idref="ref20" type="bt">3</bibl> <bibtext> Aarons G. A., Ehrhart M. G., Farahnak L. R., Sklar M., Horowitz J. (2017). Discrepancies in leader and follower ratings of transformational leadership: Relationship with organizational culture in mental health. Administration and Policy in Mental Health and Mental Health Services Research, 44(4), 480–491. https://doi.org/10.1007/s10488-015-0672-7</bibtext> </blist> <blist> <bibl id="bib4" idref="ref22" type="bt">4</bibl> <bibtext> Aarons G. A., Horowitz J. D., Dlugosz L. R., Ehrhart M. G. (2012). The role of organizational process in dissemination and implementation research. In Brownson R. C., Colditz G. A., Proctor E. K. (Eds.), Dissemination and implementation research in health: Translating science to practice (pp. 128–153). Oxford University Press.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref21" type="bt">5</bibl> <bibtext> Aarons G. A., Hurlburt M., Horwitz S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23. https://doi.org/10.1007/s10488-010-0327-7</bibtext> </blist> <blist> <bibl id="bib6" idref="ref64" type="bt">6</bibl> <bibtext> Brookman-Frazee L., Stahmer A. C. (2018). Effectiveness of a multi-level implementation strategy for ASD interventions: Study protocol for two linked cluster randomized trials. Implementation Science, 13(1). https://doi.org/10.1186/s13012-018-0757-2</bibtext> </blist> <blist> <bibl id="bib7" idref="ref31" type="bt">7</bibl> <bibtext> California Autism Professional Training and Information Network. (2017). Welcome. <ulink href="http://www.captain.ca.gov/">http://www.captain.ca.gov/</ulink></bibtext> </blist> <blist> <bibl id="bib8" idref="ref43" type="bt">8</bibl> <bibtext> Ehrhart M. G., Aarons G. A., Farahnak L. R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9(1), 157–168. https://doi.org/10.1186/s13012-014-0157-1</bibtext> </blist> <blist> <bibl id="bib9" idref="ref13" type="bt">9</bibl> <bibtext> Fixsen D., Blase K., Metz A., Van Dyke M. (2013). Statewide implementation of evidence based programs. Exceptional Children, 79(2), 213–230. https://doi.org/10.1177/001440291307900206</bibtext> </blist> <blist> <bibtext> Gibbs A. (1997). Focus groups. Social Research Update, 19(8), 1–8. https://sru.soc.surrey.ac.uk/SRU19.html</bibtext> </blist> <blist> <bibtext> Green A. E., Albanese B. J., Cafri G., Aarons G. A. (2014). Leadership, organizational climate, and working alliance in a children's mental health service system. Community Mental Health Journal, 50(7), 771–777. https://doi.org/10.1007/s10597-013-9668-5</bibtext> </blist> <blist> <bibtext> Hamilton A. B., Finley E. P. (2019). Qualitative methods in implementation research: An introduction. Psychiatry Research, 280, Article 112516. https://doi.org/10.1016/j.psychres.2019.112516</bibtext> </blist> <blist> <bibtext> Harvey G., Fitzgerald L., Fielden S., McBride A., Waterman H., Bamford D., Kislov R., Boaden R. (2011). The NIHR collaboration for leadership in applied health research and care (CLAHRC) for Greater Manchester: Combining empirical, theoretical, and experiential evidence to design and evaluate a large-scale implementation strategy. Implementation Science, 6(1), Article 96. https://doi.org/10.1186/1748-5908-6-96</bibtext> </blist> <blist> <bibtext> Hess K. L., Morrier M. J., Heflin L. J., Ivey M. L. (2008). Autism treatment survey: Services received by children with autism spectrum disorders in public school classrooms. Journal of Autism and Developmental Disorders, 38(5), 961–971. https://doi.org/10.1007/s10803-007-0470-5</bibtext> </blist> <blist> <bibtext> Kratz H. E., Stahmer A., Xie M., Marcus S. C., Pellecchia M., Locke J., Beidas R., Mandell D. S. (2019). The effect of implementation climate on program fidelity and student outcomes in autism support classrooms. Journal of Consulting and Clinical Psychology, 87(3), 270–281. https://doi.org/10.1037/ccp0000368</bibtext> </blist> <blist> <bibtext> Lasky B., Karge B. D. (2006). Meeting the needs of students with disabilities: Experience and confidence of principals. NASSP Bulletin, 90(1), 19–36. https://doi.org/10.1177%2F0192636505283950</bibtext> </blist> <blist> <bibtext> Locke J., Lawson G. M., Beidas R. S., Aarons G. A., Xie M., Lyon A. R., Stahmer A., Seidman M., Fredrick L., Oh C., Spaulding C., Dorsey S., Mandell D. (2019). Individual and organizational factors that affect implementation of evidence-based practices for children with autism in public schools: A cross-sectional observational study. Implementation Science, 14(1), Article 29. https://doi.org/10.1186/s13012-019-0877-3</bibtext> </blist> <blist> <bibtext> Lyon A. R., Cook C. R., Brown E. C., Locke J., Davis C., Ehrhart M., Aarons G. A. (2018). Assessing organizational implementation context in the education sector: Confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implementation Science, 13(1), 1–14. https://doi.org/10.1186/s13012-017-0705-6</bibtext> </blist> <blist> <bibtext> Maenner M. J., Shaw K. A., Baio J. (2020). Prevalence of autism spectrum disorder among children aged 8 years—Autism and developmental disabilities monitoring network, 11 sites, United States, 2016. MMWR Surveillance Summaries, 69(4), 1. https://doi.org/10.15585/mmwr.ss6904a1</bibtext> </blist> <blist> <bibtext> Melgarejo M., Lind T., Stadnick N. A., Helm J. L., Locke J. (2020). Strengthening capacity for implementation of evidence-based practices for autism in schools: The roles of implementation climate, school leadership, and fidelity. American Psychologist, 75(8), 1105–1115. https://doi.org/10.1037/amp0000649</bibtext> </blist> <blist> <bibtext> Merton R. K., Kendall P. L. (1946). The focused interview. American Journal of Sociology, 51(6), 541–557. https://doi.org/10.2307/2088873</bibtext> </blist> <blist> <bibtext> Novins D. K., Green A. E., Legha R. K., Aarons G. A. (2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child & Adolescent Psychiatry, 52(10), 1009–1025. https://doi.org/10.1016/j.jaac.2013.07.012</bibtext> </blist> <blist> <bibtext> Odom S. L., Sam A. M., Tomaszewski B. (2022). Factors associated with implementation of a school-based comprehensive program for students with autism. Autism, 26(3), 703–715. https://doi.org/10.1177/13623613211070340</bibtext> </blist> <blist> <bibtext> Palinkas L. A., Aarons G. A., Horwitz S., Chamberlain P., Hurlburt M., Landsverk J. (2011). Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 44–53. https://doi.org/10.1007/s10488-010-0314-z</bibtext> </blist> <blist> <bibtext> Pellecchia M., Connell J. E., Beidas R. S., Xie M., Marcus S. C., Mandell D. S. (2015). Dismantling the active ingredients of an intervention for children with autism. Journal of Autism and Developmental Disorders, 45(9), 2917–2927. https://doi.org/10.1007/s10803-015-2455-0</bibtext> </blist> <blist> <bibtext> QSR International. (2012). NVivo 10 [Computer software]. https://<ulink href="http://www.qsrinternational.com/nvivo/home">www.qsrinternational.com/nvivo/home</ulink></bibtext> </blist> <blist> <bibtext> Robins C. S., Ware N. C., dosReis S., Willging C. E., Chung J. Y., Lewis-Fernandez R. (2008). Dialogues on mixed methods and mental health services research: Anticipating challenges, building solutions. Psychiatric Services, 59, 727–731. https://doi.org/10.1176%2Fappi.ps.59.7.727</bibtext> </blist> <blist> <bibtext> Sam A. M., Cox A. W., Savage M. N., Waters V., Odom S. L. (2020). Disseminating information on evidence-based practices for children and youth with autism spectrum disorder: AFIRM. Journal of Autism and Developmental Disorders, 50(6), 1931–1940. https://doi.org/10.1007/s10803-019-03945-x</bibtext> </blist> <blist> <bibtext> Sanetti L. M. H., Collier-Meek M. A. (2019). Increasing implementation science literacy to address the research-to-practice gap in school psychology. Journal of School Psychology, 76, 33–47. https://doi.org/10.1016/j.jsp.2019.07.008</bibtext> </blist> <blist> <bibtext> Snyder T. D., De Brey C., Dillow S. A. (2019). Digest of education statistics 2017 (NCES 2018-070). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.</bibtext> </blist> <blist> <bibtext> Spillane J. (2006). Distributed leadership. Jossey-Bass.</bibtext> </blist> <blist> <bibtext> Stahmer A. C., Brookman-Frazee L., Lau A., Roesch S., Caplan B. (2022, September 9). Testing a multi-level implementation strategy for two evidence-based autism interventions: Preliminary results [Paper Presented in L. Brookman-frazee (Chair) Symposium: Utilizing the TEAMS implementation-effectiveness trial to assess the joint impact of multilevel implementation strategies and implementation context on autism EBI implementation outcomes in schools and child mental health services]. Society for Implementation Research Collaboration Conference, San Diego, CA, United States.</bibtext> </blist> <blist> <bibtext> Stahmer A. C., Suhrheinrich J., Schetter P. L., Hassrick E. M. (2018). Exploring multi-level system factors facilitating educator training and implementation of evidence-based practices (EBP): A study protocol. Implementation Science, 13, Article 3. https://doi.org/10.1186/s13012-017-0698-1</bibtext> </blist> <blist> <bibtext> Steinbrenner J. R., Hume K., Odom S. L., Morin K. L., Nowell S. W., Tomaszewski B., Szendrey S., McIntyre N. S., Yücesoy-Özkan S., Savage M. N. (2020). Evidence-based practices for children, youth, and young adults with autism. The University of North Carolina at Chapel Hill, Frank Porter Graham Child Development Institute, National Clearinghouse on Autism Evidence and Practice Review Team.</bibtext> </blist> <blist> <bibtext> Suhrheinrich J., Rieth S. R., Dickson K. S., Stahmer A. C. (2020a). Exploring associations between inner-context factors and implementation outcomes. Exceptional Children, 86(2), 155–173. https://doi.org/10.1177%2F0014402919881354</bibtext> </blist> <blist> <bibtext> Suhrheinrich J., Schetter P., England A., Melgarejo M., Nahmias A. S., Dean M., Yasuda P. (2020b). Statewide interagency collaboration to support evidence-based practice scale up: The California Autism Professional Training and Information Network (CAPTAIN). Evidence-Based Practice in Child and Adolescent Mental Health, 5, 468–482. https://doi.org/10.1080/23794925.2020.1796545</bibtext> </blist> <blist> <bibtext> Suhrheinrich J., Stahmer A. C., Reed S., Schreibman L., Reisinger E., Mandell D. (2013). Implementation challenges in translating pivotal response training into community settings. Journal of Autism and Developmental Disorders, 43(12), 2970–2976. https://doi.org/10.1007/s10803-013-1826-7</bibtext> </blist> <blist> <bibtext> Thayer A. J., Brown E. C., Cook C. R., Ehrhart M., Locke J., Davis C., Picozzi E., Aarons G., Lyon A. R. (2022). Construct validity of the school-implementation climate scale (SICS). Implementation Research and Practice, 3, 1–14. https://doi.org/10.1177/26334895221116065</bibtext> </blist> <blist> <bibtext> U.S. Department of Education. (2016, January 8). RDA: Results driven accountability. https://www2.ed.gov/about/offices/list/osers/osep/rda/index.html</bibtext> </blist> <blist> <bibtext> Weiner B. J., Belden C. M., Bergmire D. M., Johnston M. (2011). The meaning and measurement of implementation climate. Implementation Science, 6(1), Article 78. https://doi.org/10.1186/1748-5908-6-78</bibtext> </blist> <blist> <bibtext> Williams N. J., Hugh M. L., Cooney D. J., Worley J. A., Locke J. (2022). Testing a theory of implementation leadership and climate across autism evidence-based interventions of varying complexity. Behavior Therapy, 53, 900–912. https://doi.org/10.1016/j.beth.2022.03.001</bibtext> </blist> <blist> <bibtext> Wong C., Odom S. L., Hume K. A., Cox A. W., Fettig A., Kucharczyk S., Brock M. E., Plavnick J. B., Fleury V. P., Schultz T. R. (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45(7), 1951–1966. https://doi.org/10.1007/s10803-014-2351-z</bibtext> </blist> </ref> <ref id="AN0162352852-42"> <title> Footnotes </title> <blist> <bibtext> Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.</bibtext> </blist> <blist> <bibtext> Funding Funding for this work was supported by a grant from the Institute of Education Sciences, U.S. Department of Education Grant: R324A140005. Dr. Jessica Suhrheinrich is also funded through an NIMH career development award K01MH109547.</bibtext> </blist> <blist> <bibtext> ORCID iD Allison S. Nahmias</bibtext> </blist> <blist> <bibtext>Graph https://orcid.org/0000-0003-4678-0367</bibtext> </blist> </ref> <aug> <p>By Melina Melgarejo; Allison S. Nahmias; Jessica Suhrheinrich; Patricia L. Schetter; Michelle Dean; Tana Holt; Jennica Li and Aubyn C. Stahmer</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib19" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib30" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib34" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib42" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib14" firstref="ref5"></nolink> <nolink nlid="nl6" bibid="bib28" firstref="ref6"></nolink> <nolink nlid="nl7" bibid="bib25" firstref="ref7"></nolink> <nolink nlid="nl8" bibid="bib37" firstref="ref8"></nolink> <nolink nlid="nl9" bibid="bib35" firstref="ref9"></nolink> <nolink nlid="nl10" bibid="bib29" firstref="ref10"></nolink> <nolink nlid="nl11" bibid="bib17" firstref="ref11"></nolink> <nolink nlid="nl12" bibid="bib31" firstref="ref12"></nolink> <nolink nlid="nl13" bibid="bib20" firstref="ref15"></nolink> <nolink nlid="nl14" bibid="bib16" firstref="ref16"></nolink> <nolink nlid="nl15" bibid="bib40" firstref="ref17"></nolink> <nolink nlid="nl16" bibid="bib22" firstref="ref18"></nolink> <nolink nlid="nl17" bibid="bib13" firstref="ref23"></nolink> <nolink nlid="nl18" bibid="bib11" firstref="ref25"></nolink> <nolink nlid="nl19" bibid="bib15" firstref="ref26"></nolink> <nolink nlid="nl20" bibid="bib41" firstref="ref27"></nolink> <nolink nlid="nl21" bibid="bib32" firstref="ref30"></nolink> <nolink nlid="nl22" bibid="bib36" firstref="ref32"></nolink> <nolink nlid="nl23" bibid="bib24" firstref="ref34"></nolink> <nolink nlid="nl24" bibid="bib27" firstref="ref35"></nolink> <nolink nlid="nl25" bibid="bib10" firstref="ref40"></nolink> <nolink nlid="nl26" bibid="bib21" firstref="ref41"></nolink> <nolink nlid="nl27" bibid="bib33" firstref="ref42"></nolink> <nolink nlid="nl28" bibid="bib38" firstref="ref44"></nolink> <nolink nlid="nl29" bibid="bib18" firstref="ref46"></nolink> <nolink nlid="nl30" bibid="bib26" firstref="ref49"></nolink> <nolink nlid="nl31" bibid="bib12" firstref="ref50"></nolink> <nolink nlid="nl32" bibid="bib39" firstref="ref55"></nolink> <nolink nlid="nl33" bibid="bib23" firstref="ref58"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1370143
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Exploring Organizational Differences in Perceptions of Implementation Climate and Leadership in Schools: A Mixed Methods Study of Autism EBP Implementation
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Melgarejo%2C+Melina%22">Melgarejo, Melina</searchLink><br /><searchLink fieldCode="AR" term="%22Nahmias%2C+Allison+S%2E%22">Nahmias, Allison S.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-4678-0367">0000-0003-4678-0367</externalLink>)<br /><searchLink fieldCode="AR" term="%22Suhrheinrich%2C+Jessica%22">Suhrheinrich, Jessica</searchLink><br /><searchLink fieldCode="AR" term="%22Schetter%2C+Patricia+L%2E%22">Schetter, Patricia L.</searchLink><br /><searchLink fieldCode="AR" term="%22Dean%2C+Michelle%22">Dean, Michelle</searchLink><br /><searchLink fieldCode="AR" term="%22Holt%2C+Tana%22">Holt, Tana</searchLink><br /><searchLink fieldCode="AR" term="%22Li%2C+Jennica%22">Li, Jennica</searchLink><br /><searchLink fieldCode="AR" term="%22Stahmer%2C+Aubyn+C%2E%22">Stahmer, Aubyn C.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Focus+on+Autism+and+Other+Developmental+Disabilities%22"><i>Focus on Autism and Other Developmental Disabilities</i></searchLink>. Mar 2023 38(1):17-31.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications and Hammill Institute on Disabilities. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 15
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)<br />National Institute of Mental Health (NIMH) (DHHS/NIH)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R324A140005<br />K01MH109547
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Evidence+Based+Practice%22">Evidence Based Practice</searchLink><br /><searchLink fieldCode="DE" term="%22Outcomes+of+Education%22">Outcomes of Education</searchLink><br /><searchLink fieldCode="DE" term="%22Autism+Spectrum+Disorders%22">Autism Spectrum Disorders</searchLink><br /><searchLink fieldCode="DE" term="%22Children%22">Children</searchLink><br /><searchLink fieldCode="DE" term="%22Special+Education%22">Special Education</searchLink><br /><searchLink fieldCode="DE" term="%22Leadership%22">Leadership</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Environment%22">Educational Environment</searchLink><br /><searchLink fieldCode="DE" term="%22School+Personnel%22">School Personnel</searchLink><br /><searchLink fieldCode="DE" term="%22Attitudes%22">Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Implementation%22">Program Implementation</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/10883576221140149
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1088-3576<br />1538-4829
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Public programs have been charged with implementation of evidence-based practices (EBPs) to improve outcomes for children with autism. However, research indicates that scale-up of EBPs poses challenges. This study identifies perceived variables linked to effective statewide scale-up of EBPs in special education by exploring implementation climate and leadership across special education organization types (e.g., schools, districts, and regional consortiums). A simultaneous QUAL + QUAN mixed methods design was employed with the primary function of convergence and triangulation. Data were drawn from focus groups with 30 special educators and a statewide survey completed by 656 school personnel in one U.S. state. In general, perceptions of implementation climate and leadership are weak in special education organizations, with strengths at regional levels focused on special education and increased challenges at the school and district levels. Implications for practice and future research are identified.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1370143
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1370143
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/10883576221140149
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 15
        StartPage: 17
    Subjects:
      – SubjectFull: Evidence Based Practice
        Type: general
      – SubjectFull: Outcomes of Education
        Type: general
      – SubjectFull: Autism Spectrum Disorders
        Type: general
      – SubjectFull: Children
        Type: general
      – SubjectFull: Special Education
        Type: general
      – SubjectFull: Leadership
        Type: general
      – SubjectFull: Educational Environment
        Type: general
      – SubjectFull: School Personnel
        Type: general
      – SubjectFull: Attitudes
        Type: general
      – SubjectFull: Program Implementation
        Type: general
    Titles:
      – TitleFull: Exploring Organizational Differences in Perceptions of Implementation Climate and Leadership in Schools: A Mixed Methods Study of Autism EBP Implementation
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Melgarejo, Melina
      – PersonEntity:
          Name:
            NameFull: Nahmias, Allison S.
      – PersonEntity:
          Name:
            NameFull: Suhrheinrich, Jessica
      – PersonEntity:
          Name:
            NameFull: Schetter, Patricia L.
      – PersonEntity:
          Name:
            NameFull: Dean, Michelle
      – PersonEntity:
          Name:
            NameFull: Holt, Tana
      – PersonEntity:
          Name:
            NameFull: Li, Jennica
      – PersonEntity:
          Name:
            NameFull: Stahmer, Aubyn C.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 03
              Type: published
              Y: 2023
          Identifiers:
            – Type: issn-print
              Value: 1088-3576
            – Type: issn-electronic
              Value: 1538-4829
          Numbering:
            – Type: volume
              Value: 38
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Focus on Autism and Other Developmental Disabilities
              Type: main
ResultId 1