View in EDS

Machine Beats Experts: Automatic Discovery of Skill Models for Data-Driven Online Course Refinement

Saved in:
Bibliographic Details
Title: Machine Beats Experts: Automatic Discovery of Skill Models for Data-Driven Online Course Refinement
Language: English
Authors: Matsuda, Noboru, Furukawa, Tadanobu, Bier, Norman, Faloutsos, Christos, International Educational Data Mining Society
Source: International Educational Data Mining Society. 2015.
Availability: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org
Peer Reviewed: N
Page Count: 8
Publication Date: 2015
Sponsoring Agency: National Science Foundation (NSF)
Contract Number: 1418244
Document Type: Speeches/Meeting Papers
Reports - Research
Descriptors: Online Courses, Skills, Automation, Models, Data, Formative Evaluation, Correlation, Comparative Analysis
Geographic Terms: Pennsylvania
Abstract: How can we automatically determine which skills must be mastered for the successful completion of an online course? Large-scale online courses (e.g., MOOCs) often contain a broad range of contents frequently intended to be a semester's worth of materials; this breadth often makes it difficult to articulate an accurate set of skills and knowledge (i.e., a skill model, or the QMatrix). We have developed an innovative method to discover skill models from the data of online courses. Our method assumes that online courses have a pre-defined skill map for which skills are associated with formative assessment items embedded throughout the online course. Our method carefully exploits correlations between various parts of student performance, as well as in the text of assessment items, to build a superior statistical model that even outperforms human experts. To evaluate our method, we compare our method with existing methods (LFA) and human engineered skill models on three Open Learning Initiative (OLI) courses at Carnegie Mellon University. The results show that (1) our method outperforms human-engineered skill models, (2) skill models discovered by our method are interpretable, and (3) our method is remarkably faster than existing methods. These results suggest that our method provides a significant contribution to the evidence-based, iterative refinement of online courses with a promising scalability. [For complete proceedings, see ED560503.]
Abstractor: As Provided
Number of References: 19
Entry Date: 2015
Accession Number: ED560513
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED560513
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED560513
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Machine Beats Experts: Automatic Discovery of Skill Models for Data-Driven Online Course Refinement
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Matsuda%2C+Noboru%22">Matsuda, Noboru</searchLink><br /><searchLink fieldCode="AR" term="%22Furukawa%2C+Tadanobu%22">Furukawa, Tadanobu</searchLink><br /><searchLink fieldCode="AR" term="%22Bier%2C+Norman%22">Bier, Norman</searchLink><br /><searchLink fieldCode="AR" term="%22Faloutsos%2C+Christos%22">Faloutsos, Christos</searchLink><br /><searchLink fieldCode="AR" term="%22International+Educational+Data+Mining+Society%22">International Educational Data Mining Society</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Educational+Data+Mining+Society%22"><i>International Educational Data Mining Society</i></searchLink>. 2015.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 8
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2015
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Science Foundation (NSF)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: 1418244
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Online+Courses%22">Online Courses</searchLink><br /><searchLink fieldCode="DE" term="%22Skills%22">Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br /><searchLink fieldCode="DE" term="%22Models%22">Models</searchLink><br /><searchLink fieldCode="DE" term="%22Data%22">Data</searchLink><br /><searchLink fieldCode="DE" term="%22Formative+Evaluation%22">Formative Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Correlation%22">Correlation</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Pennsylvania%22">Pennsylvania</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: How can we automatically determine which skills must be mastered for the successful completion of an online course? Large-scale online courses (e.g., MOOCs) often contain a broad range of contents frequently intended to be a semester's worth of materials; this breadth often makes it difficult to articulate an accurate set of skills and knowledge (i.e., a skill model, or the QMatrix). We have developed an innovative method to discover skill models from the data of online courses. Our method assumes that online courses have a pre-defined skill map for which skills are associated with formative assessment items embedded throughout the online course. Our method carefully exploits correlations between various parts of student performance, as well as in the text of assessment items, to build a superior statistical model that even outperforms human experts. To evaluate our method, we compare our method with existing methods (LFA) and human engineered skill models on three Open Learning Initiative (OLI) courses at Carnegie Mellon University. The results show that (1) our method outperforms human-engineered skill models, (2) skill models discovered by our method are interpretable, and (3) our method is remarkably faster than existing methods. These results suggest that our method provides a significant contribution to the evidence-based, iterative refinement of online courses with a promising scalability. [For complete proceedings, see ED560503.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 19
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2015
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED560513
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED560513
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 8
    Subjects:
      – SubjectFull: Online Courses
        Type: general
      – SubjectFull: Skills
        Type: general
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Models
        Type: general
      – SubjectFull: Data
        Type: general
      – SubjectFull: Formative Evaluation
        Type: general
      – SubjectFull: Correlation
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Pennsylvania
        Type: general
    Titles:
      – TitleFull: Machine Beats Experts: Automatic Discovery of Skill Models for Data-Driven Online Course Refinement
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: International Educational Data Mining Society
      – PersonEntity:
          Name:
            NameFull: Matsuda, Noboru
      – PersonEntity:
          Name:
            NameFull: Furukawa, Tadanobu
      – PersonEntity:
          Name:
            NameFull: Bier, Norman
      – PersonEntity:
          Name:
            NameFull: Faloutsos, Christos
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 06
              Type: published
              Y: 2015
          Titles:
            – TitleFull: International Educational Data Mining Society
              Type: main
ResultId 1