Impact of Verbal Scale Labels on the Elevation and Spread of Performance Ratings.
Saved in:
| Title: | Impact of Verbal Scale Labels on the Elevation and Spread of Performance Ratings. |
|---|---|
| Authors: | Kuhlemeier, Hans (AUTHOR), Hemker, Bas (AUTHOR), van den Bergh, Huub (AUTHOR) |
| Source: | Applied Measurement in Education. Jan-Mar2013, Vol. 26 Issue 1, p16-33. 18p. 4 Charts. |
| Subjects: | National competency-based educational tests, Vocational education, Educational tests & measurements, Psychometrics |
| Geographic Terms: | Netherlands |
| Abstract: | In recent years many countries have introduced authentic performance-based assessments in their national exam systems. Teachers’ ratings of their own candidates’ performances may suffer from errors of leniency and range restriction. The goal of this study was to examine the impact of manipulating the descriptiveness, balancedness, and polarity of the rating scales on the elevation and spread of the performance ratings. The study was conducted in the field setting of a (simulated) high-stakes national exam in Dutch pre-vocational education. A total of 55 teachers rated the performances of 652 candidates (aged ±16) on four authentic performance-based tasks. Multivariate multilevel analyses found the psychometric quality of the teachers’ performance ratings to be more favorable for positively unbalanced scales than for balanced scales. Positively unbalanced rating scales yielded the lowest (i.e., least generous) and most discriminative ratings. The descriptiveness and polarity of the rating scales were of lesser importance for the rating distributions. On the basis of the findings it was decided to introduce positively unbalanced scales in the national exams for pre-vocational education. [ABSTRACT FROM AUTHOR] |
| Copyright of Applied Measurement in Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Psychology and Behavioral Sciences Collection |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | In recent years many countries have introduced authentic performance-based assessments in their national exam systems. Teachers’ ratings of their own candidates’ performances may suffer from errors of leniency and range restriction. The goal of this study was to examine the impact of manipulating the descriptiveness, balancedness, and polarity of the rating scales on the elevation and spread of the performance ratings. The study was conducted in the field setting of a (simulated) high-stakes national exam in Dutch pre-vocational education. A total of 55 teachers rated the performances of 652 candidates (aged ±16) on four authentic performance-based tasks. Multivariate multilevel analyses found the psychometric quality of the teachers’ performance ratings to be more favorable for positively unbalanced scales than for balanced scales. Positively unbalanced rating scales yielded the lowest (i.e., least generous) and most discriminative ratings. The descriptiveness and polarity of the rating scales were of lesser importance for the rating distributions. On the basis of the findings it was decided to introduce positively unbalanced scales in the national exams for pre-vocational education. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 08957347 |
| DOI: | 10.1080/08957347.2013.739425 |
