View in EDS HTML Full Text PDF Full Text

Impact of Verbal Scale Labels on the Elevation and Spread of Performance Ratings.

Saved in:
Bibliographic Details
Title: Impact of Verbal Scale Labels on the Elevation and Spread of Performance Ratings.
Authors: Kuhlemeier, Hans (AUTHOR), Hemker, Bas (AUTHOR), van den Bergh, Huub (AUTHOR)
Source: Applied Measurement in Education. Jan-Mar2013, Vol. 26 Issue 1, p16-33. 18p. 4 Charts.
Subjects: National competency-based educational tests, Vocational education, Educational tests & measurements, Psychometrics
Geographic Terms: Netherlands
Abstract: In recent years many countries have introduced authentic performance-based assessments in their national exam systems. Teachers’ ratings of their own candidates’ performances may suffer from errors of leniency and range restriction. The goal of this study was to examine the impact of manipulating the descriptiveness, balancedness, and polarity of the rating scales on the elevation and spread of the performance ratings. The study was conducted in the field setting of a (simulated) high-stakes national exam in Dutch pre-vocational education. A total of 55 teachers rated the performances of 652 candidates (aged ±16) on four authentic performance-based tasks. Multivariate multilevel analyses found the psychometric quality of the teachers’ performance ratings to be more favorable for positively unbalanced scales than for balanced scales. Positively unbalanced rating scales yielded the lowest (i.e., least generous) and most discriminative ratings. The descriptiveness and polarity of the rating scales were of lesser importance for the rating distributions. On the basis of the findings it was decided to introduce positively unbalanced scales in the national exams for pre-vocational education. [ABSTRACT FROM AUTHOR]
Copyright of Applied Measurement in Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Psychology and Behavioral Sciences Collection
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: pbh
DbLabel: Psychology and Behavioral Sciences Collection
An: 84590251
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Impact of Verbal Scale Labels on the Elevation and Spread of Performance Ratings.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Kuhlemeier%2C+Hans%22">Kuhlemeier, Hans</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Hemker%2C+Bas%22">Hemker, Bas</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22van+den+Bergh%2C+Huub%22">van den Bergh, Huub</searchLink> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Applied+Measurement+in+Education%22">Applied Measurement in Education</searchLink>. Jan-Mar2013, Vol. 26 Issue 1, p16-33. 18p. 4 Charts.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22National+competency-based+educational+tests%22">National competency-based educational tests</searchLink><br /><searchLink fieldCode="DE" term="%22Vocational+education%22">Vocational education</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+tests+%26+measurements%22">Educational tests & measurements</searchLink><br /><searchLink fieldCode="DE" term="%22Psychometrics%22">Psychometrics</searchLink>
– Name: SubjectGeographic
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Netherlands%22">Netherlands</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In recent years many countries have introduced authentic performance-based assessments in their national exam systems. Teachers’ ratings of their own candidates’ performances may suffer from errors of leniency and range restriction. The goal of this study was to examine the impact of manipulating the descriptiveness, balancedness, and polarity of the rating scales on the elevation and spread of the performance ratings. The study was conducted in the field setting of a (simulated) high-stakes national exam in Dutch pre-vocational education. A total of 55 teachers rated the performances of 652 candidates (aged ±16) on four authentic performance-based tasks. Multivariate multilevel analyses found the psychometric quality of the teachers’ performance ratings to be more favorable for positively unbalanced scales than for balanced scales. Positively unbalanced rating scales yielded the lowest (i.e., least generous) and most discriminative ratings. The descriptiveness and polarity of the rating scales were of lesser importance for the rating distributions. On the basis of the findings it was decided to introduce positively unbalanced scales in the national exams for pre-vocational education. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Applied Measurement in Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=pbh&AN=84590251
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/08957347.2013.739425
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 18
        StartPage: 16
    Subjects:
      – SubjectFull: National competency-based educational tests
        Type: general
      – SubjectFull: Vocational education
        Type: general
      – SubjectFull: Educational tests & measurements
        Type: general
      – SubjectFull: Psychometrics
        Type: general
      – SubjectFull: Netherlands
        Type: general
    Titles:
      – TitleFull: Impact of Verbal Scale Labels on the Elevation and Spread of Performance Ratings.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Kuhlemeier, Hans
      – PersonEntity:
          Name:
            NameFull: Hemker, Bas
      – PersonEntity:
          Name:
            NameFull: van den Bergh, Huub
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Text: Jan-Mar2013
              Type: published
              Y: 2013
          Identifiers:
            – Type: issn-print
              Value: 08957347
          Numbering:
            – Type: volume
              Value: 26
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Applied Measurement in Education
              Type: main
ResultId 1