Analyzing Small Sample Sizes after Disaggregation: Exploring Observation Oriented Modeling for Assessing Learning Outcomes
Saved in:
| Title: | Analyzing Small Sample Sizes after Disaggregation: Exploring Observation Oriented Modeling for Assessing Learning Outcomes |
|---|---|
| Language: | English |
| Authors: | J. Vincent Nix, Yi-Chin Wu, Lan Misty Song, Joseph D. Levy |
| Source: | Research & Practice in Assessment. 2024 19(1):72-88. |
| Availability: | Virginia Assessment Group.; Fax: 504-247-1232; e-mail: editor@rpajournal.com; Web site: http://www.rpajournal.com/ |
| Peer Reviewed: | Y |
| Page Count: | 17 |
| Publication Date: | 2024 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Sample Size, Statistical Significance, MOOCs, Geographic Location, English (Second Language), LGBTQ People, Data Analysis, Student Evaluation, Nonparametric Statistics |
| ISSN: | 2161-4210 |
| Abstract: | Traditionally, assessment professionals use analyses relying upon null hypothesis significance testing (NHST), but those tools have limitations when analyzing small samples or disaggregated data. This study used common NHST analytical techniques, compared their results, and then explored an alternative technique that perhaps allows for a more precise understanding of learning outcomes; to that end, it juxtaposed results from NHST techniques to Observation Oriented Modeling (OOM) results using data from an eight-module, massive online open course (MOOC). The OOM analysis effectively analyzed small sample sizes after disaggregation, revealing relationships between learners' locations and the disproportionate outcomes among two groups: non-native English-speakers and persons identifying as non-binary. These findings show how meaningful disaggregation and interrogation of collected data can help instructors improve pedagogy and materials, as well as serve the field at large by suggesting alternative analytical tools to glean meaningful results from difficult-to-analyze data. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1481036 |
| Database: | ERIC |
| Abstract: | Traditionally, assessment professionals use analyses relying upon null hypothesis significance testing (NHST), but those tools have limitations when analyzing small samples or disaggregated data. This study used common NHST analytical techniques, compared their results, and then explored an alternative technique that perhaps allows for a more precise understanding of learning outcomes; to that end, it juxtaposed results from NHST techniques to Observation Oriented Modeling (OOM) results using data from an eight-module, massive online open course (MOOC). The OOM analysis effectively analyzed small sample sizes after disaggregation, revealing relationships between learners' locations and the disproportionate outcomes among two groups: non-native English-speakers and persons identifying as non-binary. These findings show how meaningful disaggregation and interrogation of collected data can help instructors improve pedagogy and materials, as well as serve the field at large by suggesting alternative analytical tools to glean meaningful results from difficult-to-analyze data. |
|---|---|
| ISSN: | 2161-4210 |
