View in EDS HTML Full Text PDF Full Text

Concordance between a U.S. Educational Autism Classification and the Autism Diagnostic Observation Schedule

Saved in:
Bibliographic Details
Title: Concordance between a U.S. Educational Autism Classification and the Autism Diagnostic Observation Schedule
Language: English
Authors: Maddox, Brenna B., Rump, Keiran M., Stahmer, Aubyn C., Suhrheinrich, Jessica, Rieth, Sarah R., Nahmias, Allison S., Nuske, Heather J., Reisinger, Erica M., Crabbe, Samantha R., Bronstein, Briana, Mandell, David S.
Source: Grantee Submission. 2020.
Peer Reviewed: Y
Page Count: 12
Publication Date: 2020
Sponsoring Agency: National Institute of Mental Health (NIMH) (DHHS/NIH)
Institute of Education Sciences (ED)
Contract Number: F32MH111166
R01MH106175
1R01MH083717
R324A080195
R324A140005
Document Type: Reports - Research
Descriptors: Autism, Pervasive Developmental Disorders, Observation, Diagnostic Tests, Special Education, Eligibility, Children, Preadolescents, Classification, Cognitive Ability, Geographic Location, Intelligence Quotient
Geographic Terms: Pennsylvania (Philadelphia), California (San Diego)
Assessment and Survey Identifiers: Autism Diagnostic Observation Schedule
DOI: 10.1080/15374416.2019.1567345
Abstract: Objective: States in the United States differ in how they determine special education eligibility for autism services. Few states include an autism-specific diagnostic tool in their evaluation. In research, the Autism Diagnostic Observation Schedule (ADOS for first edition, ADOS-2 for second edition) is considered the gold-standard autism assessment. The purpose of this study was to estimate the proportion of children with an educational classification of autism who exceed the ADOS/ADOS-2 threshold for autism spectrum (concordance rate). Method: Data were drawn from four school-based studies across two sites (Philadelphia, Pennsylvania and San Diego, California). Participants comprised 627 children (ages 2-12 years; 83% male) with an autism educational classification. Analyses included 1) calculating the concordance rate between educational and ADOS/ADOS-2 classifications, and 2) estimating the associations between concordance and child's cognitive ability, study site, and ADOS/ADOS-2 administration year using logistic regression. Results: More (97.5%) San Diego participants (all assessed with the ADOS-2) met ADOS/ADOS-2 classification than did Philadelphia participants assessed with the ADOS-2 (92.2%) or ADOS (82.9%). Children assessed more recently were assessed with the ADOS-2; this group was more likely to meet ADOS/ADOS-2 classification than the group assessed a longer time ago with the ADOS. Children with higher IQ were less likely to meet ADOS/ADOS-2 classification. Conclusions: Most children with an educational classification of autism meet ADOS/ADOS-2 criteria, but results differ by site and also by ADOS version and/or recency of assessment. Educational classification may be a reasonable but imperfect measure to include children in community-based trials. [This paper was published in "Journal of Clinical Child & Adolescent Psychology" v49 n4 p469-475 2020.]
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2022
Accession Number: ED618108
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwEomNXOJBMWMu9L9agNSOkyAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDJCE_wlGPRoC6cYvDAIBEICBmq3oqMBJh2hFQ1mh73GgWK2GF09PJUtGTHhXUdWVrNmrY1iVDNUGQkT_9Dscpfv-MlS3nI3OG2AVkCrhZctCrHyj0LgW2BDj9723dU_PKAQPAOD5AQAtdezm3QijDa6EIybp3eRlrr5cKVesJMYIWu-D_3U697o4KNS7qzkboq7WnZEulJ7Lt4Hdt0nyCUSEtjznwsp07mJNogE=
Text:
  Availability: 1
  Value: <anid>AN0144655504;kyt01jul.20;2020Jul21.03:03;v2.2.500</anid> <title id="AN0144655504-1">Concordance between a U.S. Educational Autism Classification and the Autism Diagnostic Observation Schedule </title> <p>States in the United States differ in how they determine special education eligibility for autism services. Few states include an autism-specific diagnostic tool in their evaluation. In research, the Autism Diagnostic Observation Schedule (ADOS for first edition, ADOS-2 for second edition) is considered the gold-standard autism assessment. The purpose of this study was to estimate the proportion of children with an educational classification of autism who exceed the ADOS/ADOS-2 threshold for autism spectrum (concordance rate). Data were drawn from 4 school-based studies across 2 sites (Philadelphia, Pennsylvania, and San Diego, California). Participants comprised 627 children (2–12 years of age; 83% male) with an autism educational classification. Analyses included (a) calculating the concordance rate between educational and ADOS/ADOS-2 classifications and (b) estimating the associations between concordance and child's cognitive ability, study site, and ADOS/ADOS-2 administration year using logistic regression. More San Diego participants (97.5%, all assessed with the ADOS-2) met ADOS/ADOS-2 classification than did Philadelphia participants assessed with the ADOS-2 (92.2%) or ADOS (82.9%). Children assessed more recently were assessed with the ADOS-2; this group was more likely to meet ADOS/ADOS-2 classification than the group assessed longer ago with the ADOS. Children with higher IQ were less likely to meet ADOS/ADOS-2 classification. Most children with an educational classification of autism meet ADOS/ADOS-2 criteria, but results differ by site and by ADOS version and/or recency of assessment. Educational classification may be a reasonable but imperfect measure to include children in community-based trials.</p> <p>During the 2014–2015 academic year, 575,879 children ages 3 to 21 in the United States received services under the special education classification of autism (Institute of Education Sciences, National Center for Education Statistics, [<reflink idref="bib12" id="ref1">12</reflink>]). Although the Individuals with Disabilities Education Act of 1990 established autism as a separate category of children eligible for special education services, states vary in the criteria they use to identify these children (MacFarlane & Kanaya, [<reflink idref="bib16" id="ref2">16</reflink>]; Pennington, Cullinan, & Southern, [<reflink idref="bib22" id="ref3">22</reflink>]). Of note, the special education eligibility requirements for autism services are distinct from the psychiatric diagnostic criteria (American Psychiatric Association, [<reflink idref="bib3" id="ref4">3</reflink>]), and a clinical diagnosis of autism is not required or sufficient to be eligible for autism services. Compared with the psychiatric diagnostic criteria for autism, the educational classification criteria as defined by Individuals with Disabilities Education Act are less specific about core symptoms and include the additional stipulation that a child's autism-related behaviors must impair his or her educational functioning. Although some states require that a clinician provide an autism diagnosis for special education eligibility, others simply require that a clinician be included on the evaluation team; most states do not require any clinician participation in the evaluation process (MacFarlane & Kanaya, [<reflink idref="bib16" id="ref5">16</reflink>]), although they may still be involved. To determine eligibility for autism services, states commonly include assessments of intelligence, speech-language, social-developmental history, and adaptive behavior, gathered through formal testing, parent interview, and direct observation (Pennington et al., [<reflink idref="bib22" id="ref6">22</reflink>]). Few states include an autism-specific assessment tool (Pennington et al., [<reflink idref="bib22" id="ref7">22</reflink>]); when an autism-specific tool is used, it is usually a screening measure rather than a diagnostic instrument (Aiello, Ruble, & Esler, [<reflink idref="bib1" id="ref8">1</reflink>]).</p> <p>This evaluation process in schools highlights a gap between educational practice and research standards. For example, in research, the Autism Diagnostic Observation Schedule, First Edition (ADOS; Lord et al., [<reflink idref="bib13" id="ref9">13</reflink>]) or Autism Diagnostic Observation Schedule, Second Edition (ADOS-2; Lord et al., [<reflink idref="bib14" id="ref10">14</reflink>]) is considered the gold-standard observational autism assessment and is generally required for clinical characterization in peer-reviewed autism studies. Although the ADOS/ADOS-2 was not designed to be a stand-alone autism assessment, it often is used this way by researchers as confirmation of diagnosis for study inclusion.</p> <p>The ADOS/ADOS-2 is time-consuming and requires extensive training, making it relatively inaccessible in a school setting. A survey of school and clinical psychologists found that they viewed the ADOS as advantageous for capturing autism spectrum disorder–specific behaviors and standardizing the structure of behavioral observation but burdensome in terms of required resources (Akshoomoff, Corsello, & Schmidt, [<reflink idref="bib2" id="ref11">2</reflink>]).</p> <p>How many students with an educational classification of autism meet research criteria for autism on the ADOS/ADOS-2 is an important question for several reasons. First, many epidemiological studies rely on special education data to estimate the prevalence of autism (e.g., Gurney et al., [<reflink idref="bib9" id="ref12">9</reflink>]; Newschaffer, Falb, & Gurney, [<reflink idref="bib20" id="ref13">20</reflink>]). Notably, educational records form a major part of the Centers for Disease Control and Prevention's population-based surveillance system, which provides highly publicized autism prevalence estimates (Rice et al., [<reflink idref="bib23" id="ref14">23</reflink>]). In these studies, researchers do not directly assess children. Instead, conclusions are based on data provided in the educational records, sometimes augmented by records from other sources.</p> <p>Second, the accuracy of the educational classification of autism offers insight into community diagnostic and school placement practices. If many children who do not meet ADOS/ADOS-2 criteria for autism receive the educational classification of autism, it suggests the need to reevaluate these practices and consider the differences between a clinical diagnosis (which often is supported with ADOS/ADOS-2 results) and an educational classification of autism (e.g., a child's autism-related behaviors must interfere with his or her educational functioning to be eligible for the educational classification). Disparities between educational and clinical classifications may mean that children are receiving autism services when they are not appropriate, or that university-based studies include children that are very different from the population of children receiving autism services in schools.</p> <p>Third, as autism intervention research moves toward conducting larger pragmatic trials, validating autism diagnosis becomes cost prohibitive. If the educational classification of autism is highly concordant with a research diagnosis supported by the ADOS/ADOS-2, then researchers could have more confidence in the educational classification and not spend the time or money on in-depth diagnostic validation.</p> <p>The purpose of this study was to estimate the proportion of children with an educational classification of autism who exceed the ADOS/ADOS-2 threshold for autism spectrum.</p> <hd id="AN0144655504-2">Methods</hd> <p></p> <hd id="AN0144655504-3">Procedures</hd> <p>Data were drawn from three school-based studies conducted at the University of Pennsylvania, in partnership with the School District of Philadelphia, and one school-based study conducted at the University of California at San Diego (investigators are now at University of California, Davis and San Diego State University), in partnership with 17 San Diego County school districts. Philadelphia is the largest city in the state of Pennsylvania, and San Diego is the second largest city in the state of California. All studies had Institutional Review Board approval. Two of the Philadelphia studies were school-based intervention trials. The first trial took place between 2008 and 2010 and used the ADOS (Mandell et al., [<reflink idref="bib17" id="ref15">17</reflink>]). The second took place between 2015 and 2016 and used the ADOS-2 (Pellecchia et al., [<reflink idref="bib21" id="ref16">21</reflink>]). Inclusion criteria for students in these two trials included (a) being enrolled full-time in an autism-support classroom and (b) being in kindergarten, first grade, or second grade. The third Philadelphia study comprised an observational study (2014–2016) of preschool children with autism and used the ADOS-2 (Nahmias, [<reflink idref="bib19" id="ref17">19</reflink>]). Participants in this study were between the ages of 35 and 59 months and receiving community-based preschool early intervention. The San Diego study was a school-intervention trial that took place between 2012 and 2015 and used the ADOS-2 (Suhrheinrich, Rieth, Dickson, Roesch, & Stahmer, [<reflink idref="bib24" id="ref18">24</reflink>]). Participating students were 3–12 years of age with an educational classification of autism in preschool or elementary school.</p> <hd id="AN0144655504-4">Participants</hd> <p>Participants comprised 627 children (518 male [83%]; 2–12 years of age). All children had an educational classification of autism and an ADOS (<emph>n</emph> = 287) or ADOS-2 (<emph>n</emph> = 340) completed by a trained clinician or researcher. Most participants were served in special education/autism support classrooms and not inclusion settings. Most participants (<emph>n</emph> = 390) were from Philadelphia; 237 were from San Diego. See Table 1 for participant characteristics.</p> <p>TABLE 1 Demographic Information by Site</p> <p> <ephtml> <table><thead><tr><td /><td>Pennsylvania Sample<sup>a</sup><italic>n</italic> (%) or <italic>M</italic> (<italic>SD</italic>); range</td><td>California Sample<sup>b</sup><italic>n</italic> (%) or <italic>M</italic> (<italic>SD</italic>); range</td></tr></thead><tbody><tr><td>Male</td><td>315 (80.8%)</td><td>203 (85.7%)</td></tr><tr><td>Race/Ethnicity<sup>c,</sup>**</td><td /><td /></tr><tr><td> Black or African American</td><td>186 (47.7%)</td><td>12 (5.1%)</td></tr><tr><td> White</td><td>111 (28.5%)</td><td>121 (51.1%)</td></tr><tr><td> Hispanic or Latino</td><td>56 (14.4%)</td><td>80 (33.8%)</td></tr><tr><td> Asian</td><td>20 (5.1%)</td><td>18 (7.6%)</td></tr><tr><td> American Indian</td><td>0 (0.0%)</td><td>4 (1.7%)</td></tr><tr><td> Multiracial</td><td>18 (4.6%)</td><td>18 (7.6%)</td></tr><tr><td> Other</td><td>1 (0.3%)</td><td>13 (5.5%)</td></tr><tr><td>Age (years)**</td><td>6.47 (1.21); 2.92–9.50</td><td>5.70 (2.06); 3.00–12.25</td></tr><tr><td>IQ<sup>d,</sup>*</td><td>61.81 (20.22); 30–113</td><td>65.92 (19.57); 28–128</td></tr></tbody></table> </ephtml> </p> <p>1 <sups>a</sups><emph>n</emph> = 390.</p> <ulist> <item>2 <sups>b</sups><emph>n</emph> = 237.</item> <item>3 <sups>c</sups>Race/ethnicity data were available for a subset of the total sample (<emph>n</emph> = 336 of Pennsylvania sample and <emph>n</emph> = 186 of California sample).</item> <item>4 <sups>d</sups>IQ measured by the Differential Ability Scales, Second Edition, General Conceptual Ability Standard Score (<emph>n</emph> = 357 of Pennsylvania sample and <emph>n</emph> = 123 of California sample) or the Mullen Scales of Early Learning (Mullen) Early Learning Composite Standard Score (<emph>n</emph> = 33 of Pennsylvania sample and <emph>n</emph> = 114 of California sample).</item> <item>5 *<emph>p</emph> <.05. **<emph>p</emph> <.001 for between-group comparisons.</item> </ulist> <hd id="AN0144655504-5">Measures/Variables</hd> <p>The dependent variable was concordance between educational classification and ADOS/ADOS-2 classification. The following ADOS or ADOS-2 modules were used to characterize the sample: Module 1 (for children without phrase speech), Module 2 (for children with phrase speech who are not verbally fluent), and Module 3 (for children with fluent language). The ADOS or ADOS-2 was coded by a research reliable ADOS clinician or researcher. The ADOS/ADOS-2 and cognitive assessment (described next) took place at either the child's school or home, in the quietest, least distracting space available.</p> <p>Independent variables included study site (Pennsylvania vs. California), ADOS/ADOS-2 module administered, year of ADOS/ADOS-2 administration, child's gender, child's race/ethnicity, child's age, and child's cognitive ability. Cognitive ability was measured using the Differential Ability Scales, Second Edition (DAS-II; Elliott, [<reflink idref="bib7" id="ref19">7</reflink>]; both Early Years and School-Age batteries) or the Mullen Scales of Early Learning (Mullen, [<reflink idref="bib18" id="ref20">18</reflink>]), depending on the child's developmental level. Because some of the sample completed the DAS-II and some completed the Mullen, we used the DAS-II General Conceptual Ability standard score and Mullen Early Learning Composite standard score to estimate cognitive ability. These two measures are highly correlated in children with neurodevelopmental disorders (Farmer, Golden, & Thurm, [<reflink idref="bib8" id="ref21">8</reflink>]).</p> <hd id="AN0144655504-6">Data Analyses</hd> <p>All analyses were conducted using SPSS version 24. First, we calculated means and standard deviations or percentages as appropriate for each variable of interest. Between-site comparisons were conducted using independent samples <emph>t</emph> tests or chi-square tests. Second, we calculated the proportion of children with an educational classification of autism who exceeded the validated threshold for "autism spectrum" or "autism" on the respective ADOS/ADOS-2 modules (Lord et al., [<reflink idref="bib13" id="ref22">13</reflink>], [<reflink idref="bib14" id="ref23">14</reflink>]). Based on (a) differences in state-specific special education eligibility and (b) differences between the Pennsylvania and California samples on demographic variables (Table 1), we conducted this analysis separately for the two sites. In addition, given that the earliest Pennsylvania study used the first edition of the ADOS, we separated this study from the later Pennsylvania studies, which used the ADOS-2. Pairwise follow-up tests were conducted for this analysis with the Holm's sequential Bonferroni method to correct for multiple comparisons (Holm, [<reflink idref="bib10" id="ref24">10</reflink>]). Third, we estimated the associations between covariates of interest—study site, ADOS/ADOS-2 module administered, year of ADOS/ADOS-2 administration, child's gender, child's race/ethnicity, child's age, child's cognitive ability—and concordance between educational classification and ADOS/ADOS-2 classification using logistic regression. We first estimated bivariate associations between each covariate and concordance. Variables significant at <emph>p</emph> <.2 were included in the adjusted logistic regression model (Hosmer & Lemeshow, [<reflink idref="bib11" id="ref25">11</reflink>]). Given that ADOS edition and study year are interdependent variables, we included only study year in this analysis.</p> <hd id="AN0144655504-7">Results</hd> <p>Table 2 presents the proportion of participants who exceeded the ADOS/ADOS-2 threshold for "autism spectrum" or "autism." The chi-square test comparing the three groups (Pennsylvania participants with ADOS data, Pennsylvania participants with ADOS-2 data, California participants with ADOS-2 data) on concordance between educational classification and ADOS/ADOS-2 classification was statistically significant, χ<sups>2</sups>(<reflink idref="bib2" id="ref26">2</reflink>) = 31.08, <emph>p</emph> <.001. Post hoc pairwise comparisons using Fisher's exact tests and Holm-Bonferroni adjusted alphas revealed that a greater percentage (97.5%) of California participants (all with ADOS-2) were concordant, compared to 92.2% of Pennsylvania participants with ADOS-2 data (<emph>p</emph> =.04) and to 82.9% of Pennsylvania participants with ADOS data (<emph>p</emph> <.001). The Pennsylvania participants with ADOS-2 data showed greater concordance than the Pennsylvania participants with ADOS data (<emph>p</emph> =.02).</p> <p>TABLE 2 ADOS/ADOS-2 Data by Site and ADOS Edition</p> <p> <ephtml> <table><thead><tr><td /><td>PA ADOS<sup>a</sup></td><td>PA ADOS-2<sup>b</sup></td><td>CA ADOS-2<sup>c</sup></td></tr></thead><tbody><tr><td>ADOS/ADOS-2 Total Score**: <italic>M</italic> (<italic>SD</italic>); range</td><td>11.99 (4.76); 0–22</td><td>15.91 (5.37); 1–25</td><td>16.97 (4.94); 2–27</td></tr><tr><td>ADOS/ADOS-2 Module: <italic>n</italic> (%)</td><td /><td /><td /></tr><tr><td> Module 1</td><td>127 (44.3%)</td><td>57 (55.3%)</td><td>122 (51.5%)</td></tr><tr><td> Module 2</td><td>108 (37.6%)</td><td>34 (33.0%)</td><td>79 (33.3%)</td></tr><tr><td> Module 3</td><td>52 (18.1%)</td><td>12 (11.7%)</td><td>36 (15.2%)</td></tr><tr><td>ADOS/ADOS-2 Classification**: <italic>n</italic> (%)</td><td /><td /><td /></tr><tr><td> Nonspectrum</td><td>49 (17.1%)</td><td>8 (7.8%)</td><td>6 (2.5%)</td></tr><tr><td> Autism spectrum</td><td>105 (36.6%)</td><td>9 (8.7%)</td><td>21 (8.9%)</td></tr><tr><td> Autism</td><td>133 (46.3%)</td><td>86 (83.5%)</td><td>210 (88.6%)</td></tr></tbody></table> </ephtml> </p> <ulist> <item>6 <sups>a</sups><emph>n</emph> = 287.</item> <item>7 <sups>b</sups><emph>n</emph> = 103.</item> <item>8 <sups>c</sups><emph>n</emph> = 237.</item> <item>9 **<emph>p</emph> <.001 for between-group comparisons, with the Pennsylvania (PA) Autism Diagnostic Observation Schedule, first edition (ADOS) group having a significantly lower ADOS total score and significantly lower concordance rate than the Pennsylvania Autism Diagnostic Observation Schedule, second edition (ADOS-2) group or California (CA) ADOS-2 group. There was no significant difference between the two ADOS-2 groups on ADOS-2 total score. The PA ADOS-2 group had a significantly lower concordance rate than the CA ADOS-2 group. Pairwise comparisons were corrected with the Holm's sequential Bonferroni method (Holm, [<reflink idref="bib10" id="ref27">10</reflink>]).</item> </ulist> <p>Table 3 presents the logistic regression results. Three variables were statistically significantly associated with concordance between educational classification and ADOS/ADOS-2 classification. Children from California were more likely than children from Pennsylvania to meet ADOS/ADOS-2 criteria (odds ratio [<emph>OR</emph>] = 5.79), 95% CI [2.31, 14.48], <emph>p</emph> <.001. Children with higher IQ were less likely to meet criteria (<emph>OR</emph> =.98 per IQ point), 95% CI [.96,.99], <emph>p</emph> =.02. Children assessed more recently were more likely to meet criteria (<emph>OR</emph> = 1.15 per year), 95% CI [1.01, 1.30], <emph>p</emph> =.03.</p> <p>TABLE 3 Logistic Regression Results</p> <p> <ephtml> <table><thead><tr><td>Variables</td><td>β (<italic>SE</italic>)</td><td>Wald</td><td>Odds Ratio</td><td>95% CI</td></tr></thead><tbody><tr><td>State**</td><td>1.76 (.47)</td><td>14.09</td><td>5.79</td><td>[2.31, 14.48]</td></tr><tr><td>IQ*</td><td>−.02 (.01)</td><td>5.32</td><td>0.98</td><td>[.96,.99]</td></tr><tr><td>ADOS module</td><td /><td>2.54</td><td /><td /></tr><tr><td> Module 2</td><td>−.47 (.39)</td><td>1.45</td><td>0.62</td><td>[.29, 1.35]</td></tr><tr><td> Module 3</td><td>−.80 (.50)</td><td>2.52</td><td>0.45</td><td>[.17, 1.21]</td></tr><tr><td>ADOS year*</td><td>.14 (.06)</td><td>4.69</td><td>1.15</td><td>[1.01, 1.30]</td></tr></tbody></table> </ephtml> </p> <ulist> <item>10 <emph>Note</emph>. CI = confidence interval; State = Pennsylvania (coded as 0) or California (coded as 1); IQ = Differential Ability Scales, Second Edition, General Conceptual Ability Standard Score or Mullen Scales of Early Learning, Early Learning Composite Standard Score; ADOS Module 1 = reference group; ADOS year = year of Diagnostic Observation Schedule administration.</item> <item>11 *<emph>p</emph> <.05. **<emph>p</emph> <.001 for between-group comparisons.</item> </ulist> <hd id="AN0144655504-8">Discussion</hd> <p>In our sample of 627 children receiving special education services for autism, 90% met criteria for "autism spectrum" or "autism" on the ADOS/ADOS-2. Although this high percentage is encouraging, it is important to note that concordance significantly differed between the two sites. Close to 98% of the children in California met ADOS-2 criteria, compared with only 92.2% of Pennsylvania participants with ADOS-2 data and 82.9% of Pennsylvania participants with ADOS data. This difference could be explained by interstate differences in how schools determine eligibility for autism services. For example, under Pennsylvania state regulations, a school psychologist is required to participate in the evaluation for autism; this requirement is not included in California state regulations. This finding also could be explained by the fact that California has a more developed system of autism services; California is home to the earliest behavioral treatment studies (Lovaas, [<reflink idref="bib15" id="ref28">15</reflink>]) and an established network of regional centers that connect individuals with autism and their families to services. Because our study focused only on positive predictive value, and because there usually is a trade-off between sensitivity and specificity, it is possible that California has a higher false negative rate than Pennsylvania.</p> <p>Another variable associated with concordance was the child's cognitive ability. Children with higher IQ scores were less likely to meet criteria on the ADOS/ADOS-2. Special education evaluation teams may be more likely to misclassify children with average or above-average IQ who present with other conditions (e.g., attention deficit/hyperactivity disorder and anxiety) as having autism. The symptoms of conditions such as attention deficit/hyperactivity disorder and anxiety can mirror some of those of autism (American Psychiatric Association, [<reflink idref="bib3" id="ref29">3</reflink>]), and it may take more sophisticated diagnostic practices than those currently used in education settings to differentiate them. In addition, previous research has found that among students with low ADOS scores (indicating fewer or less severe autism symptoms), parents rated their children as more socially impaired than teachers rated them (Azad, Reisinger, Xie, & Mandell, [<reflink idref="bib4" id="ref30">4</reflink>]). Perhaps parents of less cognitively and socially impaired children, including children who fall below the ADOS threshold, are still concerned about receiving services for their children's behavioral difficulties and therefore strongly advocate for a special education classification.</p> <p>The third factor that significantly affected concordance was the year of ADOS/ADOS-2 administration. Children assessed more recently were more likely to meet criteria. Our sample was pulled from four studies spanning from 2008 to 2016. We have three hypotheses to explain improved concordance over time. First, this finding may be related to moving from the ADOS to the ADOS-2 in 2012. The ADOS-2 boasts improved sensitivity and specificity (Lord et al., [<reflink idref="bib14" id="ref31">14</reflink>]), and we found that children assessed with the ADOS had a significantly lower concordance than the children assessed with the ADOS-2. Second, as autism services become more sophisticated, special education evaluation practices may have improved. A competing hypothesis relates to the fact that educational services for children with autism cost 3 times those for typically developing students (Buescher, Cidav, Knapp, & Mandell, [<reflink idref="bib6" id="ref32">6</reflink>]). As the number of children diagnosed with autism increases, school systems may become more stringent in their classification practices in an attempt to control the number of children in this category.</p> <p>Based on recent Department of Education statistics, the number of U.S. students ages 3 to 21 with an educational classification of autism has increased more than threefold from the 2004–2005 to 2014–2015 school year (Institute of Education Sciences, National Center for Education Statistics, [<reflink idref="bib12" id="ref33">12</reflink>]). This increase means that more children are receiving special education services for autism in their schools, which is good news for many families. School is often the most feasible treatment setting these children. The link between an educational classification and school-based services can raise difficult ethical issues for clinicians and researchers when results from a clinical evaluation do not match a child's educational classification of autism. For example, if a community clinician determines that a child does not meet diagnostic criteria for autism, but this child receives school-based autism services that the family perceives as beneficial, the clinician and family grapple with issues related to disclosure about the clinical evaluation results and possible loss of services. This tension is one reason why our research team does not exclude children with low ADOS scores from our school-based autism studies; we do not want to interfere with their current services. It is important to note that the reverse situation also can happen and cause stress for families, when a child carries a community diagnosis of autism, but results from the school evaluation do not support special education services.</p> <p>Several study limitations deserve mention. First, we relied solely on the ADOS/ADOS-2 for diagnostic categorization. Children with low ADOS/ADOS-2 scores still may meet diagnostic criteria for autism based on a more thorough assessment and expert clinician opinion (Lord et al., [<reflink idref="bib13" id="ref34">13</reflink>], [<reflink idref="bib14" id="ref35">14</reflink>]). Second, the ADOS/ADOS-2 provides a snapshot of a child's current behaviors; it is possible that children who do not meet ADOS/ADOS-2 criteria at one point in time would have meet criteria previously (when assessed for educational classification). The ADOS/ADOS-2 is not a perfect tool, and it was not designed to be a stand-alone autism assessment; however, it is often used this way by researchers as confirmation of diagnosis for study inclusion. These two limitations may have biased results by reducing the concordance rate. Third, we were unable to fully disentangle the variables of study year, ADOS edition (first or second), and study site because all of the ADOS data came from one Philadelphia study that spanned from 2008 to 2010, whereas the other three studies took place in 2012 or later and used the ADOS-2. Thus, we are unable to determine whether the improvement in concordance in more recent years is a secular or test effect; however, the improvement is encouraging, regardless of the cause. Last, the ADOS/ADOS-2 was administered in the child's school or home, which may be less controlled than a clinical setting. For the school administrations, the child's parent was not usually present, which is part of standard administration of Modules 1 and 2. The ADOS/ADOS-2 clinician relied on a teacher or paraprofessional familiar to the child when possible, but the lack of a parent was less than ideal.</p> <p>Despite these limitations, this study has key implications for autism research and services. For epidemiological researchers, estimates of autism prevalence based on one state or even several states may not generalize to the nation. Differences among states may be due in large part to different evaluation practices, and may contribute to the dramatic discrepancy in autism prevalence rates between states, based on Centers for Disease Control and Prevention data (e.g., 1 in 34 children in New Jersey vs. 1 in 76 children in Arkansas; Baio et al., [<reflink idref="bib5" id="ref36">5</reflink>]). The between-site differences found here do not support broad reliance on educational classification for research purposes. However, based on project data, concordance may vary by state. In California, researchers may place a higher degree of confidence in the educational classification of autism. In Pennsylvania, the concordance rate is not as rigorous, but it is encouraging that this rate is improving over time.</p> <p>This study also has important implications for service systems. The results suggest that on the whole, the large majority of children with an educational autism classification meet research criteria based on the ADOS-2. However, a subset of children who do not meet ADOS/ADOS-2 criteria for autism may be receiving special education services under this category.</p> <ref id="AN0144655504-9"> <title> References </title> <blist> <bibl id="bib1" idref="ref8" type="bt">1</bibl> <bibtext> Aiello, R., Ruble, L., & Esler, A. (2017). National study of school psychologists' use of evidence-based assessment in autism spectrum disorder. Journal of Applied School Psychology, 33, 67 – 88. doi: 10.1080/15377903.2016.1236307</bibtext> </blist> <blist> <bibl id="bib2" idref="ref11" type="bt">2</bibl> <bibtext> Akshoomoff, N., Corsello, C., & Schmidt, H. (2006). The role of the autism diagnostic observation schedule in the assessment of autism spectrum disorders in school and community settings. The California School Psychologist, 11, 7 – 19. doi: 10.1007/BF03341111</bibtext> </blist> <blist> <bibl id="bib3" idref="ref4" type="bt">3</bibl> <bibtext> American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC : Author.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref30" type="bt">4</bibl> <bibtext> Azad, G., Reisinger, E., Xie, M., & Mandell, D. (2016). Parent and teacher concordance on the social responsiveness scale for children with autism. School Mental Health, 8, 368 – 376. doi: 10.1007/s12310-015-9168-6</bibtext> </blist> <blist> <bibl id="bib5" idref="ref36" type="bt">5</bibl> <bibtext> Baio, J., Wiggins, L., Christensen, D. L., Maenner, M. J., Daniels, J., Warren, Z., ... Dowling, N. F. (2018). Prevalence of autism spectrum disorder among children aged 8 years — Autism and developmental disabilities monitoring network, 11 Sites, United States, 2014. Morbidity and Mortality Weekly Report Surveillance Summaries, 67 (6), 1 – 23. doi: 10.15585/mmwr.ss6706a1</bibtext> </blist> <blist> <bibl id="bib6" idref="ref32" type="bt">6</bibl> <bibtext> Buescher, A. V., Cidav, Z., Knapp, M., & Mandell, D. S. (2014). Costs of autism spectrum disorders in the United Kingdom and the United States. JAMA pediatrics, 168, 721 – 728. doi: 10.1001/jamapediatrics.2014.210</bibtext> </blist> <blist> <bibl id="bib7" idref="ref19" type="bt">7</bibl> <bibtext> Elliott, C. D. (2007). Differential Ability Scales, Second Edition (DAS-II). San Antonio, TX : Harcourt Assessment.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref21" type="bt">8</bibl> <bibtext> Farmer, C., Golden, C., & Thurm, A. (2016). Concurrent validity of the differential ability scales, second edition with the mullen scales of early learning in young children with and without neurodevelopmental disorders. Child Neuropsychology, 22, 556 – 569. doi: 10.1080/09297049.2015.1020775</bibtext> </blist> <blist> <bibl id="bib9" idref="ref12" type="bt">9</bibl> <bibtext> Gurney, J. G., Fritz, M. S., Ness, K. K., Sievers, P., Newschaffer, C. J., & Shapiro, E. G. (2003). Analysis of prevalence trends of autism spectrum disorder in Minnesota. Archives of Pediatrics & Adolescent Medicine, 157, 622 – 627. doi: 10.1001/archpedi.157.7.622</bibtext> </blist> <blist> <bibtext> Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65 – 70.</bibtext> </blist> <blist> <bibtext> Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression (2nd ed). New York, NY : John Wiley & Sons, Inc.</bibtext> </blist> <blist> <bibtext> Institute of Education Sciences, National Center for Education Statistics. (2016). Children 3 to 21 years old served under Individuals with Disabilities Education Act (IDEA), Part B, by type of disability. Retrieved from https://nces.ed.gov/programs/digest/d16/tables/dt16_204.30.asp</bibtext> </blist> <blist> <bibtext> Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavore, P. C., ... Rutter, M. (2000). The Autism diagnostic observation schedule–generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30, 205 – 223. doi: 10.1023/A:1005592401947</bibtext> </blist> <blist> <bibtext> Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. L. (2012). Autism diagnostic observation schedule (2nd ed.). Torrance, CA : Western Psychological Services.</bibtext> </blist> <blist> <bibtext> Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3 – 9. doi: 10.1037/0022-006X.55.1.3</bibtext> </blist> <blist> <bibtext> MacFarlane, J. R., & Kanaya, T. (2009). What does it mean to be autistic: Inter-state variation in special education criteria for autism services? Journal of Child and Family Studies, 18, 662 – 669. doi: 10.1007/s10826-009-9268</bibtext> </blist> <blist> <bibtext> Mandell, D. S., Stahmer, A. C., Shin, S., Xie, M., Reisinger, E., & Marcus, S. C. (2013). The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention. Autism, 17, 281 – 295. doi: 10.1177/1362361312473666</bibtext> </blist> <blist> <bibtext> Mullen, E. M. (1995). Mullen scales of early learning. Circle Pines, MN : American Guidance Service.</bibtext> </blist> <blist> <bibtext> Nahmias, A. S. (2017). Community-based early intervention for children with autism spectrum disorder (Order No. 10615988) (Available from ProQuest Dissertations & Theses A&I). (1955179401).</bibtext> </blist> <blist> <bibtext> Newschaffer, C. J., Falb, M. D., & Gurney, J. G. (2005). National autism prevalence trends from United States special education data. Pediatrics, 115, e277 – e282. doi: 10.1542/peds.2004-1958</bibtext> </blist> <blist> <bibtext> Pellecchia, M., Beidas, R. S., Marcus, S. C., Fishman, J., Kimberly, J. R., Cannuscio, C. C., ... Mandell, D. S. (2016). Study protocol: Implementation of a computer-assisted intervention for autism in schools: A hybrid type II cluster randomized effectiveness-implementation trial. Implementation Science, 11, 154. doi: 10.1186/s13012-016-0513-4</bibtext> </blist> <blist> <bibtext> Pennington, M. L., Cullinan, D., & Southern, L. B. (2014). Defining autism: Variability in state education agency definitions of and evaluations for autism spectrum disorders. Autism Research and Treatment, 2014, 1 – 8. doi: 10.1155/2014/327271</bibtext> </blist> <blist> <bibtext> Rice, C. E., Baio, J., Van Naarden Braun, K., Doernberg, N., Meaney, F. J., & Kirby, R. S. (2007). A public health collaboration for the surveillance of autism spectrum disorders. Pediatric and Perinatal Epidemiology, 21, 179 – 190. doi: 10.1111/j.1365-3016.2007.00801.x</bibtext> </blist> <blist> <bibtext> Suhrheinrich, J., Rieth, S. R., Dickson, K. S., Roesch, S., & Stahmer, A. C. (2018). Classroom pivotal response teaching: Teacher training outcomes of a community efficacy trial. Manuscript in revision.</bibtext> </blist> </ref> <aug> <p>By Brenna B. Maddox; Keiran M. Rump; Aubyn C. Stahmer; Jessica Suhrheinrich; Sarah R. Rieth; Allison S. Nahmias; Heather J. Nuske; Erica M. Reisinger; Samantha R. Crabbe; Briana Bronstein and David S. Mandell</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib12" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib16" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib22" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib13" firstref="ref9"></nolink> <nolink nlid="nl5" bibid="bib14" firstref="ref10"></nolink> <nolink nlid="nl6" bibid="bib20" firstref="ref13"></nolink> <nolink nlid="nl7" bibid="bib23" firstref="ref14"></nolink> <nolink nlid="nl8" bibid="bib17" firstref="ref15"></nolink> <nolink nlid="nl9" bibid="bib21" firstref="ref16"></nolink> <nolink nlid="nl10" bibid="bib19" firstref="ref17"></nolink> <nolink nlid="nl11" bibid="bib24" firstref="ref18"></nolink> <nolink nlid="nl12" bibid="bib18" firstref="ref20"></nolink> <nolink nlid="nl13" bibid="bib10" firstref="ref24"></nolink> <nolink nlid="nl14" bibid="bib11" firstref="ref25"></nolink> <nolink nlid="nl15" bibid="bib15" firstref="ref28"></nolink>
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED618108
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED618108
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Concordance between a U.S. Educational Autism Classification and the Autism Diagnostic Observation Schedule
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Maddox%2C+Brenna+B%2E%22">Maddox, Brenna B.</searchLink><br /><searchLink fieldCode="AR" term="%22Rump%2C+Keiran+M%2E%22">Rump, Keiran M.</searchLink><br /><searchLink fieldCode="AR" term="%22Stahmer%2C+Aubyn+C%2E%22">Stahmer, Aubyn C.</searchLink><br /><searchLink fieldCode="AR" term="%22Suhrheinrich%2C+Jessica%22">Suhrheinrich, Jessica</searchLink><br /><searchLink fieldCode="AR" term="%22Rieth%2C+Sarah+R%2E%22">Rieth, Sarah R.</searchLink><br /><searchLink fieldCode="AR" term="%22Nahmias%2C+Allison+S%2E%22">Nahmias, Allison S.</searchLink><br /><searchLink fieldCode="AR" term="%22Nuske%2C+Heather+J%2E%22">Nuske, Heather J.</searchLink><br /><searchLink fieldCode="AR" term="%22Reisinger%2C+Erica+M%2E%22">Reisinger, Erica M.</searchLink><br /><searchLink fieldCode="AR" term="%22Crabbe%2C+Samantha+R%2E%22">Crabbe, Samantha R.</searchLink><br /><searchLink fieldCode="AR" term="%22Bronstein%2C+Briana%22">Bronstein, Briana</searchLink><br /><searchLink fieldCode="AR" term="%22Mandell%2C+David+S%2E%22">Mandell, David S.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Grantee+Submission%22"><i>Grantee Submission</i></searchLink>. 2020.
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 12
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2020
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Institute of Mental Health (NIMH) (DHHS/NIH)<br />Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: F32MH111166<br />R01MH106175<br />1R01MH083717<br />R324A080195<br />R324A140005
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Autism%22">Autism</searchLink><br /><searchLink fieldCode="DE" term="%22Pervasive+Developmental+Disorders%22">Pervasive Developmental Disorders</searchLink><br /><searchLink fieldCode="DE" term="%22Observation%22">Observation</searchLink><br /><searchLink fieldCode="DE" term="%22Diagnostic+Tests%22">Diagnostic Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Special+Education%22">Special Education</searchLink><br /><searchLink fieldCode="DE" term="%22Eligibility%22">Eligibility</searchLink><br /><searchLink fieldCode="DE" term="%22Children%22">Children</searchLink><br /><searchLink fieldCode="DE" term="%22Preadolescents%22">Preadolescents</searchLink><br /><searchLink fieldCode="DE" term="%22Classification%22">Classification</searchLink><br /><searchLink fieldCode="DE" term="%22Cognitive+Ability%22">Cognitive Ability</searchLink><br /><searchLink fieldCode="DE" term="%22Geographic+Location%22">Geographic Location</searchLink><br /><searchLink fieldCode="DE" term="%22Intelligence+Quotient%22">Intelligence Quotient</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Pennsylvania+%28Philadelphia%29%22">Pennsylvania (Philadelphia)</searchLink><br /><searchLink fieldCode="DE" term="%22California+%28San+Diego%29%22">California (San Diego)</searchLink>
– Name: SubjectThesaurus
  Label: Assessment and Survey Identifiers
  Group: Su
  Data: <searchLink fieldCode="SU" term="%22Autism+Diagnostic+Observation+Schedule%22">Autism Diagnostic Observation Schedule</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/15374416.2019.1567345
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Objective: States in the United States differ in how they determine special education eligibility for autism services. Few states include an autism-specific diagnostic tool in their evaluation. In research, the Autism Diagnostic Observation Schedule (ADOS for first edition, ADOS-2 for second edition) is considered the gold-standard autism assessment. The purpose of this study was to estimate the proportion of children with an educational classification of autism who exceed the ADOS/ADOS-2 threshold for autism spectrum (concordance rate). Method: Data were drawn from four school-based studies across two sites (Philadelphia, Pennsylvania and San Diego, California). Participants comprised 627 children (ages 2-12 years; 83% male) with an autism educational classification. Analyses included 1) calculating the concordance rate between educational and ADOS/ADOS-2 classifications, and 2) estimating the associations between concordance and child's cognitive ability, study site, and ADOS/ADOS-2 administration year using logistic regression. Results: More (97.5%) San Diego participants (all assessed with the ADOS-2) met ADOS/ADOS-2 classification than did Philadelphia participants assessed with the ADOS-2 (92.2%) or ADOS (82.9%). Children assessed more recently were assessed with the ADOS-2; this group was more likely to meet ADOS/ADOS-2 classification than the group assessed a longer time ago with the ADOS. Children with higher IQ were less likely to meet ADOS/ADOS-2 classification. Conclusions: Most children with an educational classification of autism meet ADOS/ADOS-2 criteria, but results differ by site and also by ADOS version and/or recency of assessment. Educational classification may be a reasonable but imperfect measure to include children in community-based trials. [This paper was published in "Journal of Clinical Child & Adolescent Psychology" v49 n4 p469-475 2020.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2022
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED618108
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED618108
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/15374416.2019.1567345
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
    Subjects:
      – SubjectFull: Autism
        Type: general
      – SubjectFull: Pervasive Developmental Disorders
        Type: general
      – SubjectFull: Observation
        Type: general
      – SubjectFull: Diagnostic Tests
        Type: general
      – SubjectFull: Special Education
        Type: general
      – SubjectFull: Eligibility
        Type: general
      – SubjectFull: Children
        Type: general
      – SubjectFull: Preadolescents
        Type: general
      – SubjectFull: Classification
        Type: general
      – SubjectFull: Cognitive Ability
        Type: general
      – SubjectFull: Geographic Location
        Type: general
      – SubjectFull: Intelligence Quotient
        Type: general
      – SubjectFull: Pennsylvania (Philadelphia)
        Type: general
      – SubjectFull: California (San Diego)
        Type: general
      – SubjectFull: Autism Diagnostic Observation Schedule
        Type: general
    Titles:
      – TitleFull: Concordance between a U.S. Educational Autism Classification and the Autism Diagnostic Observation Schedule
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Maddox, Brenna B.
      – PersonEntity:
          Name:
            NameFull: Rump, Keiran M.
      – PersonEntity:
          Name:
            NameFull: Stahmer, Aubyn C.
      – PersonEntity:
          Name:
            NameFull: Suhrheinrich, Jessica
      – PersonEntity:
          Name:
            NameFull: Rieth, Sarah R.
      – PersonEntity:
          Name:
            NameFull: Nahmias, Allison S.
      – PersonEntity:
          Name:
            NameFull: Nuske, Heather J.
      – PersonEntity:
          Name:
            NameFull: Reisinger, Erica M.
      – PersonEntity:
          Name:
            NameFull: Crabbe, Samantha R.
      – PersonEntity:
          Name:
            NameFull: Bronstein, Briana
      – PersonEntity:
          Name:
            NameFull: Mandell, David S.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2020
          Titles:
            – TitleFull: Grantee Submission
              Type: main
ResultId 1