View in EDS

A Human-Machine Hybrid Peer Grading Framework for SPOCs

Saved in:
Bibliographic Details
Title: A Human-Machine Hybrid Peer Grading Framework for SPOCs
Language: English
Authors: Han, Yong, Wu, Wenjun, Ji, Suozhao, Zhang, Lijun, Zhang, Hui
Source: International Educational Data Mining Society. 2019.
Availability: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org
Peer Reviewed: Y
Page Count: 6
Publication Date: 2019
Document Type: Speeches/Meeting Papers
Reports - Descriptive
Descriptors: Peer Evaluation, Grading, Online Courses, Computer Assisted Testing, Man Machine Systems, Bayesian Statistics, True Scores
Abstract: Peer-grading is commonly adopted by instructors as an effective assessment method for MOOCs (Massive Open Online Courses) and SPOCs (Small Private online course). For solving the problems brought by varied skill levels and attitudes of online students, statistical models have been proposed to improve the fairness and accuracy of peer-grading. However, these models fail to deliver accurate inference in the SPOCs scenario because affinity among students may seriously affect the objectivity and reliability of students in the peer-assessment process. To address this problem, this paper proposes a human-machine hybrid peer-grading framework, including an automatic grader to ensure reasonable peer grades before the Bayesian models are utilized to infer the true scores. This framework can significantly eliminate the severely biased grades by those undutiful students, and thus improve the accuracy of the true-score estimation in the Bayesian peer-grading models. Both simulated and real peer-grading datasets in our experiments demonstrate the effectiveness of this new framework for SPOCs. [For the full proceedings, see ED599096.]
Abstractor: As Provided
Entry Date: 2019
Accession Number: ED599175
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED599175
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED599175
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A Human-Machine Hybrid Peer Grading Framework for SPOCs
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Han%2C+Yong%22">Han, Yong</searchLink><br /><searchLink fieldCode="AR" term="%22Wu%2C+Wenjun%22">Wu, Wenjun</searchLink><br /><searchLink fieldCode="AR" term="%22Ji%2C+Suozhao%22">Ji, Suozhao</searchLink><br /><searchLink fieldCode="AR" term="%22Zhang%2C+Lijun%22">Zhang, Lijun</searchLink><br /><searchLink fieldCode="AR" term="%22Zhang%2C+Hui%22">Zhang, Hui</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Educational+Data+Mining+Society%22"><i>International Educational Data Mining Society</i></searchLink>. 2019.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 6
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2019
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Descriptive
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Peer+Evaluation%22">Peer Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Grading%22">Grading</searchLink><br /><searchLink fieldCode="DE" term="%22Online+Courses%22">Online Courses</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Man+Machine+Systems%22">Man Machine Systems</searchLink><br /><searchLink fieldCode="DE" term="%22Bayesian+Statistics%22">Bayesian Statistics</searchLink><br /><searchLink fieldCode="DE" term="%22True+Scores%22">True Scores</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Peer-grading is commonly adopted by instructors as an effective assessment method for MOOCs (Massive Open Online Courses) and SPOCs (Small Private online course). For solving the problems brought by varied skill levels and attitudes of online students, statistical models have been proposed to improve the fairness and accuracy of peer-grading. However, these models fail to deliver accurate inference in the SPOCs scenario because affinity among students may seriously affect the objectivity and reliability of students in the peer-assessment process. To address this problem, this paper proposes a human-machine hybrid peer-grading framework, including an automatic grader to ensure reasonable peer grades before the Bayesian models are utilized to infer the true scores. This framework can significantly eliminate the severely biased grades by those undutiful students, and thus improve the accuracy of the true-score estimation in the Bayesian peer-grading models. Both simulated and real peer-grading datasets in our experiments demonstrate the effectiveness of this new framework for SPOCs. [For the full proceedings, see ED599096.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2019
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED599175
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED599175
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 6
    Subjects:
      – SubjectFull: Peer Evaluation
        Type: general
      – SubjectFull: Grading
        Type: general
      – SubjectFull: Online Courses
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Man Machine Systems
        Type: general
      – SubjectFull: Bayesian Statistics
        Type: general
      – SubjectFull: True Scores
        Type: general
    Titles:
      – TitleFull: A Human-Machine Hybrid Peer Grading Framework for SPOCs
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Han, Yong
      – PersonEntity:
          Name:
            NameFull: Wu, Wenjun
      – PersonEntity:
          Name:
            NameFull: Ji, Suozhao
      – PersonEntity:
          Name:
            NameFull: Zhang, Lijun
      – PersonEntity:
          Name:
            NameFull: Zhang, Hui
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 07
              Type: published
              Y: 2019
          Titles:
            – TitleFull: International Educational Data Mining Society
              Type: main
ResultId 1