Impact of Design Effects in Large-Scale District and State Assessments
Saved in:
| Title: | Impact of Design Effects in Large-Scale District and State Assessments |
|---|---|
| Language: | English |
| Authors: | Phillips, Gary W. |
| Source: | Applied Measurement in Education. 2015 28(1):33-47. |
| Availability: | Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals |
| Peer Reviewed: | Y |
| Page Count: | 15 |
| Publication Date: | 2015 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | State Programs, Sampling, Research Design, Error of Measurement, Testing Programs, Statistical Significance, School Districts, Probability, Sample Size, Response Rates (Questionnaires), Weighted Scores, Item Analysis, Equated Scores, Item Response Theory, Testing Problems, Evaluation Methods, Evaluation Problems, Experimenter Characteristics, Test Reliability, Test Validity, Group Testing |
| DOI: | 10.1080/08957347.2014.973561 |
| ISSN: | 0895-7347 |
| Abstract: | This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling errors, in turn, results in unanticipated instability in the testing program and an increase in Type I errors in significance tests. This is especially true when the standard error of equating is underestimated. The problem is caused by the typical district and state practice of using nonprobability cluster-sampling procedures, such as convenience, purposeful, and quota sampling, then calculating statistics and standard errors as if the samples were simple random samples. |
| Abstractor: | As Provided |
| Number of References: | 21 |
| Entry Date: | 2015 |
| Accession Number: | EJ1048933 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling errors, in turn, results in unanticipated instability in the testing program and an increase in Type I errors in significance tests. This is especially true when the standard error of equating is underestimated. The problem is caused by the typical district and state practice of using nonprobability cluster-sampling procedures, such as convenience, purposeful, and quota sampling, then calculating statistics and standard errors as if the samples were simple random samples. |
|---|---|
| ISSN: | 0895-7347 |
| DOI: | 10.1080/08957347.2014.973561 |
