A Human-Machine Hybrid Peer Grading Framework for SPOCs
Saved in:
| Title: | A Human-Machine Hybrid Peer Grading Framework for SPOCs |
|---|---|
| Language: | English |
| Authors: | Han, Yong, Wu, Wenjun, Ji, Suozhao, Zhang, Lijun, Zhang, Hui |
| Source: | International Educational Data Mining Society. 2019. |
| Availability: | International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org |
| Peer Reviewed: | Y |
| Page Count: | 6 |
| Publication Date: | 2019 |
| Document Type: | Speeches/Meeting Papers Reports - Descriptive |
| Descriptors: | Peer Evaluation, Grading, Online Courses, Computer Assisted Testing, Man Machine Systems, Bayesian Statistics, True Scores |
| Abstract: | Peer-grading is commonly adopted by instructors as an effective assessment method for MOOCs (Massive Open Online Courses) and SPOCs (Small Private online course). For solving the problems brought by varied skill levels and attitudes of online students, statistical models have been proposed to improve the fairness and accuracy of peer-grading. However, these models fail to deliver accurate inference in the SPOCs scenario because affinity among students may seriously affect the objectivity and reliability of students in the peer-assessment process. To address this problem, this paper proposes a human-machine hybrid peer-grading framework, including an automatic grader to ensure reasonable peer grades before the Bayesian models are utilized to infer the true scores. This framework can significantly eliminate the severely biased grades by those undutiful students, and thus improve the accuracy of the true-score estimation in the Bayesian peer-grading models. Both simulated and real peer-grading datasets in our experiments demonstrate the effectiveness of this new framework for SPOCs. [For the full proceedings, see ED599096.] |
| Abstractor: | As Provided |
| Entry Date: | 2019 |
| Accession Number: | ED599175 |
| Database: | ERIC |
| Abstract: | Peer-grading is commonly adopted by instructors as an effective assessment method for MOOCs (Massive Open Online Courses) and SPOCs (Small Private online course). For solving the problems brought by varied skill levels and attitudes of online students, statistical models have been proposed to improve the fairness and accuracy of peer-grading. However, these models fail to deliver accurate inference in the SPOCs scenario because affinity among students may seriously affect the objectivity and reliability of students in the peer-assessment process. To address this problem, this paper proposes a human-machine hybrid peer-grading framework, including an automatic grader to ensure reasonable peer grades before the Bayesian models are utilized to infer the true scores. This framework can significantly eliminate the severely biased grades by those undutiful students, and thus improve the accuracy of the true-score estimation in the Bayesian peer-grading models. Both simulated and real peer-grading datasets in our experiments demonstrate the effectiveness of this new framework for SPOCs. [For the full proceedings, see ED599096.] |
|---|
