View in EDS

Designing Crowdcritique Systems for Formative Feedback

Saved in:
Bibliographic Details
Title: Designing Crowdcritique Systems for Formative Feedback
Language: English
Authors: Easterday, Matthew W., Rees Lewis, Daniel, Gerber, Elizabeth M.
Source: International Journal of Artificial Intelligence in Education. Sep 2017 27(3):623-663.
Availability: Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: service-ny@springer.com; Web site: http://www.springerlink.com
Peer Reviewed: Y
Page Count: 41
Publication Date: 2017
Sponsoring Agency: National Science Foundation (NSF)
Contract Number: IIS1320693
Document Type: Journal Articles
Reports - Research
Descriptors: Intelligent Tutoring Systems, Formative Evaluation, Feedback (Response), Novices, Problem Solving, Observation, Interviews, Surveys, Grounded Theory, Social Media, Computer Mediated Communication, Scaffolding (Teaching Technique), Design
DOI: 10.1007/s40593-016-0125-9
ISSN: 1560-4292
Abstract: Intelligent tutors based on expert systems often struggle to provide formative feedback on complex, ill-defined problems where answers are unknown. Hybrid crowdsourcing systems that combine the intelligence of multiple novices in face-to-face settings might provide an alternate approach for providing intelligent formative feedback. The purpose of this study was to develop empirically grounded design principles for crowdcritique systems that provide intelligent formative feedback on complex, ill-defined problems. In this design research project, we iteratively developed and tested a crowdcritique system through 3 studies of 43 novice problem solvers in 3 formal and informal learning environments. We collected observations, interviews, and surveys and used a grounded theory approach to develop and test socio-technical design principles for crowdcritique systems. The project found that to provide formative feedback on ill-defined problems, crowdcritique systems should provide a combination of technical features including: quick invite tools; formative framing; a public, near-synchronous social media interface; critique scaffolds; "like" system; community hashtags; analysis tools and "to do" lists; along with social practices including: prep/write-first/write-last script and critique training. Such a system creates a dual-channel conversation that increases the volume of quality critique by grounding comments, scaffolding and recording critique, and reducing production blocking. Such a design provides the benefits of both face-to-face critique and computer-support in both formal and informal learning environments while reducing the orchestration burden on instructors.
Abstractor: As Provided
Number of References: 93
Entry Date: 2017
Accession Number: EJ1148535
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1148535
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Designing Crowdcritique Systems for Formative Feedback
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Easterday%2C+Matthew+W%2E%22">Easterday, Matthew W.</searchLink><br /><searchLink fieldCode="AR" term="%22Rees+Lewis%2C+Daniel%22">Rees Lewis, Daniel</searchLink><br /><searchLink fieldCode="AR" term="%22Gerber%2C+Elizabeth+M%2E%22">Gerber, Elizabeth M.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Journal+of+Artificial+Intelligence+in+Education%22"><i>International Journal of Artificial Intelligence in Education</i></searchLink>. Sep 2017 27(3):623-663.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: service-ny@springer.com; Web site: http://www.springerlink.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 41
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2017
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Science Foundation (NSF)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: IIS1320693
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Intelligent+Tutoring+Systems%22">Intelligent Tutoring Systems</searchLink><br /><searchLink fieldCode="DE" term="%22Formative+Evaluation%22">Formative Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Novices%22">Novices</searchLink><br /><searchLink fieldCode="DE" term="%22Problem+Solving%22">Problem Solving</searchLink><br /><searchLink fieldCode="DE" term="%22Observation%22">Observation</searchLink><br /><searchLink fieldCode="DE" term="%22Interviews%22">Interviews</searchLink><br /><searchLink fieldCode="DE" term="%22Surveys%22">Surveys</searchLink><br /><searchLink fieldCode="DE" term="%22Grounded+Theory%22">Grounded Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Social+Media%22">Social Media</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Mediated+Communication%22">Computer Mediated Communication</searchLink><br /><searchLink fieldCode="DE" term="%22Scaffolding+%28Teaching+Technique%29%22">Scaffolding (Teaching Technique)</searchLink><br /><searchLink fieldCode="DE" term="%22Design%22">Design</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s40593-016-0125-9
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1560-4292
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Intelligent tutors based on expert systems often struggle to provide formative feedback on complex, ill-defined problems where answers are unknown. Hybrid crowdsourcing systems that combine the intelligence of multiple novices in face-to-face settings might provide an alternate approach for providing intelligent formative feedback. The purpose of this study was to develop empirically grounded design principles for crowdcritique systems that provide intelligent formative feedback on complex, ill-defined problems. In this design research project, we iteratively developed and tested a crowdcritique system through 3 studies of 43 novice problem solvers in 3 formal and informal learning environments. We collected observations, interviews, and surveys and used a grounded theory approach to develop and test socio-technical design principles for crowdcritique systems. The project found that to provide formative feedback on ill-defined problems, crowdcritique systems should provide a combination of technical features including: quick invite tools; formative framing; a public, near-synchronous social media interface; critique scaffolds; "like" system; community hashtags; analysis tools and "to do" lists; along with social practices including: prep/write-first/write-last script and critique training. Such a system creates a dual-channel conversation that increases the volume of quality critique by grounding comments, scaffolding and recording critique, and reducing production blocking. Such a design provides the benefits of both face-to-face critique and computer-support in both formal and informal learning environments while reducing the orchestration burden on instructors.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 93
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2017
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1148535
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1148535
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s40593-016-0125-9
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 41
        StartPage: 623
    Subjects:
      – SubjectFull: Intelligent Tutoring Systems
        Type: general
      – SubjectFull: Formative Evaluation
        Type: general
      – SubjectFull: Feedback (Response)
        Type: general
      – SubjectFull: Novices
        Type: general
      – SubjectFull: Problem Solving
        Type: general
      – SubjectFull: Observation
        Type: general
      – SubjectFull: Interviews
        Type: general
      – SubjectFull: Surveys
        Type: general
      – SubjectFull: Grounded Theory
        Type: general
      – SubjectFull: Social Media
        Type: general
      – SubjectFull: Computer Mediated Communication
        Type: general
      – SubjectFull: Scaffolding (Teaching Technique)
        Type: general
      – SubjectFull: Design
        Type: general
    Titles:
      – TitleFull: Designing Crowdcritique Systems for Formative Feedback
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Easterday, Matthew W.
      – PersonEntity:
          Name:
            NameFull: Rees Lewis, Daniel
      – PersonEntity:
          Name:
            NameFull: Gerber, Elizabeth M.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 09
              Type: published
              Y: 2017
          Identifiers:
            – Type: issn-print
              Value: 1560-4292
          Numbering:
            – Type: volume
              Value: 27
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: International Journal of Artificial Intelligence in Education
              Type: main
ResultId 1