View in EDS

'The Wisdom of Crowds': When Teacher Judgments Outperform Word-Frequency as a Predictor of Students' Vocabulary Knowledge

Saved in:
Bibliographic Details
Title: 'The Wisdom of Crowds': When Teacher Judgments Outperform Word-Frequency as a Predictor of Students' Vocabulary Knowledge
Language: English
Authors: Pablo Robles-García (ORCID 0000-0003-4780-8552), Jeffrey Stewart (ORCID 0000-0002-3350-3160), Christopher Nicklin (ORCID 0000-0002-8945-0678), Joseph P. Vitta (ORCID 0000-0002-5711-969X), Stuart McLean (ORCID 0000-0002-7035-378X), Brandon Kramer (ORCID 0000-0003-3910-0810)
Source: Language Teaching Research. 2026 30(4):2085-2109.
Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Peer Reviewed: Y
Page Count: 25
Publication Date: 2026
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Higher Education, Undergraduate Students, College Faculty, Second Language Learning, Spanish, English (Second Language), Word Frequency, Vocabulary, Vocabulary Skills, Prediction, Accuracy, Teacher Attitudes, Word Lists
DOI: 10.1177/13621688231176067
ISSN: 1362-1688
1477-0954
Abstract: This study investigated the effectiveness of word-frequency and teacher judgments in determining students' vocabulary knowledge and compared the predictive powers of both approaches when estimating vocabulary knowledge. Twenty-nine second language (L2) Spanish teachers were asked to predict how likely their students would know words from a 216-word Yes/No test that measures knowledge of the first 3,000 words in Spanish. The accuracy of their responses was compared with the results of 1,075 L2 Spanish students who completed the same test. To examine if the results could generalize to other L2 settings, 394 L2 English students completed a 70-word Yes/No test that measures knowledge of the first 14,000 words in English, and 15 L2 English language instructors attempted to predict which words would or would not be recognized. Results showed that for both language contexts, (1) the median teacher rater could assess students' vocabulary knowledge with an accuracy roughly comparable to frequency, (2) the combination of teachers' judgments displayed a stronger relationship with students' performance on the vocabulary test than frequency, since the average of three or more teachers' ratings improved upon frequency when examined with 1,000 bootstrapped samples, and (3) using teacher judgments and frequency together did not substantially improve the prediction of students' vocabulary knowledge.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1502954
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1502954
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: 'The Wisdom of Crowds': When Teacher Judgments Outperform Word-Frequency as a Predictor of Students' Vocabulary Knowledge
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Pablo+Robles-García%22">Pablo Robles-García</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-4780-8552">0000-0003-4780-8552</externalLink>)<br /><searchLink fieldCode="AR" term="%22Jeffrey+Stewart%22">Jeffrey Stewart</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-3350-3160">0000-0002-3350-3160</externalLink>)<br /><searchLink fieldCode="AR" term="%22Christopher+Nicklin%22">Christopher Nicklin</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-8945-0678">0000-0002-8945-0678</externalLink>)<br /><searchLink fieldCode="AR" term="%22Joseph+P%2E+Vitta%22">Joseph P. Vitta</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-5711-969X">0000-0002-5711-969X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Stuart+McLean%22">Stuart McLean</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-7035-378X">0000-0002-7035-378X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Brandon+Kramer%22">Brandon Kramer</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3910-0810">0000-0003-3910-0810</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Language+Teaching+Research%22"><i>Language Teaching Research</i></searchLink>. 2026 30(4):2085-2109.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 25
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2026
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22College+Faculty%22">College Faculty</searchLink><br /><searchLink fieldCode="DE" term="%22Second+Language+Learning%22">Second Language Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Spanish%22">Spanish</searchLink><br /><searchLink fieldCode="DE" term="%22English+%28Second+Language%29%22">English (Second Language)</searchLink><br /><searchLink fieldCode="DE" term="%22Word+Frequency%22">Word Frequency</searchLink><br /><searchLink fieldCode="DE" term="%22Vocabulary%22">Vocabulary</searchLink><br /><searchLink fieldCode="DE" term="%22Vocabulary+Skills%22">Vocabulary Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Prediction%22">Prediction</searchLink><br /><searchLink fieldCode="DE" term="%22Accuracy%22">Accuracy</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Attitudes%22">Teacher Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Word+Lists%22">Word Lists</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/13621688231176067
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1362-1688<br />1477-0954
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This study investigated the effectiveness of word-frequency and teacher judgments in determining students' vocabulary knowledge and compared the predictive powers of both approaches when estimating vocabulary knowledge. Twenty-nine second language (L2) Spanish teachers were asked to predict how likely their students would know words from a 216-word Yes/No test that measures knowledge of the first 3,000 words in Spanish. The accuracy of their responses was compared with the results of 1,075 L2 Spanish students who completed the same test. To examine if the results could generalize to other L2 settings, 394 L2 English students completed a 70-word Yes/No test that measures knowledge of the first 14,000 words in English, and 15 L2 English language instructors attempted to predict which words would or would not be recognized. Results showed that for both language contexts, (1) the median teacher rater could assess students' vocabulary knowledge with an accuracy roughly comparable to frequency, (2) the combination of teachers' judgments displayed a stronger relationship with students' performance on the vocabulary test than frequency, since the average of three or more teachers' ratings improved upon frequency when examined with 1,000 bootstrapped samples, and (3) using teacher judgments and frequency together did not substantially improve the prediction of students' vocabulary knowledge.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1502954
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1502954
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/13621688231176067
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 25
        StartPage: 2085
    Subjects:
      – SubjectFull: Higher Education
        Type: general
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: College Faculty
        Type: general
      – SubjectFull: Second Language Learning
        Type: general
      – SubjectFull: Spanish
        Type: general
      – SubjectFull: English (Second Language)
        Type: general
      – SubjectFull: Word Frequency
        Type: general
      – SubjectFull: Vocabulary
        Type: general
      – SubjectFull: Vocabulary Skills
        Type: general
      – SubjectFull: Prediction
        Type: general
      – SubjectFull: Accuracy
        Type: general
      – SubjectFull: Teacher Attitudes
        Type: general
      – SubjectFull: Word Lists
        Type: general
    Titles:
      – TitleFull: 'The Wisdom of Crowds': When Teacher Judgments Outperform Word-Frequency as a Predictor of Students' Vocabulary Knowledge
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Pablo Robles-García
      – PersonEntity:
          Name:
            NameFull: Jeffrey Stewart
      – PersonEntity:
          Name:
            NameFull: Christopher Nicklin
      – PersonEntity:
          Name:
            NameFull: Joseph P. Vitta
      – PersonEntity:
          Name:
            NameFull: Stuart McLean
      – PersonEntity:
          Name:
            NameFull: Brandon Kramer
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 05
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 1362-1688
            – Type: issn-electronic
              Value: 1477-0954
          Numbering:
            – Type: volume
              Value: 30
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Language Teaching Research
              Type: main
ResultId 1