View in EDS

The Effects of Assessment Design on Academic Dishonesty, Learner Engagement, and Certification Rates in MOOCs

Saved in:
Bibliographic Details
Title: The Effects of Assessment Design on Academic Dishonesty, Learner Engagement, and Certification Rates in MOOCs
Language: English
Authors: Alexandron, Giora (ORCID 0000-0003-2676-6912), Wiltrout, Mary Ellen, Berg, Aviram, Gershon, Sa'ar Karp, Ruipérez-Valiente, José A.
Source: Journal of Computer Assisted Learning. Feb 2023 39(1):141-153.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 13
Publication Date: 2023
Document Type: Journal Articles
Reports - Research
Descriptors: MOOCs, Evaluation, Models, Learner Engagement, Certification, Incidence, Cheating, Competence, Time Management, Test Construction, Graduation Rate, Minimum Competency Testing
DOI: 10.1111/jcal.12733
ISSN: 0266-4909
1365-2729
Abstract: Background: Massive Open Online Courses (MOOCs) have touted the idea of democratizing education, but soon enough, this utopian idea collided with the reality of finding sustainable business models. In addition, the promise of harnessing interactive and social web technologies to promote meaningful learning was only partially successful. And finally, studies demonstrated that many learners exploit the anonymity and feedback to earn certificates unethically. Thus, establishing MOOC pedagogical models that balance open access, meaningful learning, and trustworthy assessment remains a challenge that is crucial for the field to achieve its goals. Objectives: This study analysed the influence of an MOOC assessment model, denoted the Competency Exam (CE), on learner engagement, the level of cheating, and certification rates. At its core, this model separates learning from for-credit assessment, and it was introduced by the MITx Biology course team in 2016. Methods: We applied a learning analytics methodology to the clickstream data of the verified learners (N = 559) from four consecutive runs of an Introductory Biology MOOC offered through edX. The analysis used novel algorithms for measuring the level of cheating and learner engagement, which were developed in the previous studies. Results and Conclusions: On the positive side, the CE model reduced cheating and did not reduce learner engagement with the main learning materials -- videos and formative assessment items. On the negative side, it led to procrastination, and certification rates were lower. Implications: First, the results shed light on the fundamental connection between incentive design and learner behaviour. Second, the CE provides MOOC designers with an 'analytically verified' model to reduce cheating without compromising on open access. Third, our methodology provides a novel means for measuring cheating and learner engagement in MOOCs.
Abstractor: As Provided
Entry Date: 2023
Accession Number: EJ1362072
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1362072
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: The Effects of Assessment Design on Academic Dishonesty, Learner Engagement, and Certification Rates in MOOCs
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Alexandron%2C+Giora%22">Alexandron, Giora</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-2676-6912">0000-0003-2676-6912</externalLink>)<br /><searchLink fieldCode="AR" term="%22Wiltrout%2C+Mary+Ellen%22">Wiltrout, Mary Ellen</searchLink><br /><searchLink fieldCode="AR" term="%22Berg%2C+Aviram%22">Berg, Aviram</searchLink><br /><searchLink fieldCode="AR" term="%22Gershon%2C+Sa'ar+Karp%22">Gershon, Sa'ar Karp</searchLink><br /><searchLink fieldCode="AR" term="%22Ruipérez-Valiente%2C+José+A%2E%22">Ruipérez-Valiente, José A.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Computer+Assisted+Learning%22"><i>Journal of Computer Assisted Learning</i></searchLink>. Feb 2023 39(1):141-153.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 13
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22MOOCs%22">MOOCs</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation%22">Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Models%22">Models</searchLink><br /><searchLink fieldCode="DE" term="%22Learner+Engagement%22">Learner Engagement</searchLink><br /><searchLink fieldCode="DE" term="%22Certification%22">Certification</searchLink><br /><searchLink fieldCode="DE" term="%22Incidence%22">Incidence</searchLink><br /><searchLink fieldCode="DE" term="%22Cheating%22">Cheating</searchLink><br /><searchLink fieldCode="DE" term="%22Competence%22">Competence</searchLink><br /><searchLink fieldCode="DE" term="%22Time+Management%22">Time Management</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Construction%22">Test Construction</searchLink><br /><searchLink fieldCode="DE" term="%22Graduation+Rate%22">Graduation Rate</searchLink><br /><searchLink fieldCode="DE" term="%22Minimum+Competency+Testing%22">Minimum Competency Testing</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/jcal.12733
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0266-4909<br />1365-2729
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Massive Open Online Courses (MOOCs) have touted the idea of democratizing education, but soon enough, this utopian idea collided with the reality of finding sustainable business models. In addition, the promise of harnessing interactive and social web technologies to promote meaningful learning was only partially successful. And finally, studies demonstrated that many learners exploit the anonymity and feedback to earn certificates unethically. Thus, establishing MOOC pedagogical models that balance open access, meaningful learning, and trustworthy assessment remains a challenge that is crucial for the field to achieve its goals. Objectives: This study analysed the influence of an MOOC assessment model, denoted the Competency Exam (CE), on learner engagement, the level of cheating, and certification rates. At its core, this model separates learning from for-credit assessment, and it was introduced by the MITx Biology course team in 2016. Methods: We applied a learning analytics methodology to the clickstream data of the verified learners (N = 559) from four consecutive runs of an Introductory Biology MOOC offered through edX. The analysis used novel algorithms for measuring the level of cheating and learner engagement, which were developed in the previous studies. Results and Conclusions: On the positive side, the CE model reduced cheating and did not reduce learner engagement with the main learning materials -- videos and formative assessment items. On the negative side, it led to procrastination, and certification rates were lower. Implications: First, the results shed light on the fundamental connection between incentive design and learner behaviour. Second, the CE provides MOOC designers with an 'analytically verified' model to reduce cheating without compromising on open access. Third, our methodology provides a novel means for measuring cheating and learner engagement in MOOCs.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1362072
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1362072
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/jcal.12733
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 13
        StartPage: 141
    Subjects:
      – SubjectFull: MOOCs
        Type: general
      – SubjectFull: Evaluation
        Type: general
      – SubjectFull: Models
        Type: general
      – SubjectFull: Learner Engagement
        Type: general
      – SubjectFull: Certification
        Type: general
      – SubjectFull: Incidence
        Type: general
      – SubjectFull: Cheating
        Type: general
      – SubjectFull: Competence
        Type: general
      – SubjectFull: Time Management
        Type: general
      – SubjectFull: Test Construction
        Type: general
      – SubjectFull: Graduation Rate
        Type: general
      – SubjectFull: Minimum Competency Testing
        Type: general
    Titles:
      – TitleFull: The Effects of Assessment Design on Academic Dishonesty, Learner Engagement, and Certification Rates in MOOCs
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Alexandron, Giora
      – PersonEntity:
          Name:
            NameFull: Wiltrout, Mary Ellen
      – PersonEntity:
          Name:
            NameFull: Berg, Aviram
      – PersonEntity:
          Name:
            NameFull: Gershon, Sa'ar Karp
      – PersonEntity:
          Name:
            NameFull: Ruipérez-Valiente, José A.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 02
              Type: published
              Y: 2023
          Identifiers:
            – Type: issn-print
              Value: 0266-4909
            – Type: issn-electronic
              Value: 1365-2729
          Numbering:
            – Type: volume
              Value: 39
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Journal of Computer Assisted Learning
              Type: main
ResultId 1