Professors Cede Grading Power to Outsiders--Even Computers
Saved in:
| Title: | Professors Cede Grading Power to Outsiders--Even Computers |
|---|---|
| Language: | English |
| Authors: | Young, Jeffrey R. |
| Source: | Education Digest: Essential Readings Condensed for Quick Review. Dec 2011 77(4):24-28. |
| Availability: | Prakken Publications. 832 Phoenix Drive, P.O. Box 8623, Ann Arbor, MI 48108. Tel: 734-975-2800; Fax: 734-975-2787; Web site: http://www.eddigest.com/ |
| Peer Reviewed: | N |
| Page Count: | 5 |
| Publication Date: | 2011 |
| Document Type: | Journal Articles Reports - Descriptive |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Grading, Evaluators, Outsourcing, Computers, Test Scoring Machines, Virtual Universities, Public Colleges |
| Geographic Terms: | Florida |
| ISSN: | 0013-127X |
| Abstract: | The best way to eliminate grade inflation is to take professors out of the grading process: Replace them with professional evaluators who never meet students and don't worry that students will punish harsh grades with poor reviews. That's the argument made by leaders of Western Governors University, which has hired 300 adjunct professors who do nothing but grade student work. Western Governors is not the only institution reassessing grading. A few others, including the University of Central Florida, now outsource scoring of some essay tests to computers. Their software can grade essays thanks to improvements in artificial-intelligence techniques. Software has no emotional biases, either, and one instructor says machines have proved more fair and balanced in grading than humans. Naturally, the standard grading model has plenty of defenders, including some who argue that claims of grade inflation are exaggerated. The current system forges a nurturing relationship between instructor and student and gives individualized attention that no robot or stranger could give, this argument goes. But the efforts at Western Governors and Central Florida could change that relationship, and point to ways to pop any grade-inflation bubble. |
| Abstractor: | ERIC |
| Entry Date: | 2012 |
| Access URL: | https://www.eddigest.com/index.php |
| Accession Number: | EJ964247 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwEoF7gKZipNIHoCtmWQ70M6AAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDAb57r4LEHRdHlGTywIBEICBmiBOoT9p83D1graojdQuZpp9qZfhN0X3P6TBGua5JEvPl1k4YMTG2v0evrXII0SipDaajR12xoBnJkES1EOk_xkD_igXv5FNENMGUQNWwWlqmF24DUftY3UTJUP1c0ZDrIGOUrji0YcY0Thi-Vbbqz8vV_WyQzX_EdTyIhCEQAutrfC319jF2knnd98jpiKdPMAWpn7zk4FlzO0= Text: Availability: 1 Value: <anid>AN0067664250;edd01dec.11;2011Dec07.07:01;v2.2.460</anid> <title id="AN0067664250-1">Professors Cede Grading Power to Outsiders - Even Computers </title> <p>The best way to eliminate grade inflation is to take professors out of the grading process: Replace them with professional evaluators who never meet students and don't worry that students will punish harsh grades with poor reviews. That's the argument made by leaders of Western Governors University, which has hired 300 adjunct professors who do nothing but grade student work.</p> <p>"They think like assessors, not professors," says Diane Johnson, who is in charge of the university's graders. "The evaluators have no contact with the students. They don't know them. They don't know what color they are, what they look like, or where they live. Because of that, there is no temptation to skew results in any way other than to judge the students' work."</p> <p>Western Governors is not the only institution reassessing grading. A few others, including the University of Central Florida, now outsource scoring of some essay tests to computers. Their software can grade essays thanks to improvements in artificial-intelligence techniques. Software has no emotional biases, either, and one instructor says machines have proved more fair and balanced in grading than humans.</p> <p>These efforts raise the question: What if professors aren't that good at grading? What if the model of giving instructors full control over grades is fundamentally flawed? As more observers call for evidence of college value in an era of rising tuition costs, game-changing models like these get serious consideration.</p> <p>Professors do score poorly when it comes to fair grading, according to a study published in July in the journal Teachers College Record. After crunching numbers on decades' worth of grade reports from about 135 colleges, researchers found that average grades have risen for 30 years, and that A is now the most common grade given at most colleges. Authors Stuart Rojstaczer and Christopher Healy argue that a "consumer-based approach" to higher education has created subtle incentives for professors to give higher marks than deserved.</p> <p>Naturally, the standard grading model has plenty of defenders, including some who argue that claims of grade inflation are exaggerated. The current system forges a nurturing relationship between instructor and student and gives individualized attention that no robot or stranger could give, this argument goes.</p> <p>But the efforts at Western Governors and Central Florida could change that relationship, and point to ways to pop any grade-inflation bubble.</p> <hd id="AN0067664250-2">An Army of Graders</hd> <p>Western Governors is an institution that turns the typical university structure on its head. Western Governors is entirely online. Technically it doesn't offer courses; instead it provides mentors who help students prepare for high-stakes homework assignments designed by a team of professional test-makers to prove competence in various subject areas.</p> <p>The idea is that as long as students can leap all of those hurdles, they deserve degrees, whether or not they've participated in any traditional teaching experience. The model is called "competency-based education."</p> <p>Designers of Western Governors do not intend to compete with any traditional institution. The online university throws a lifeline to nontraditional students who can't make it to those campuses.</p> <p>Western Governors splits the role of the traditional professor into two jobs. Instructional duties fall to a group the university calls "course mentors," who help students master material. The graders, or evaluators, step in once the homework is filed, with the mind-set of, "OK, the teaching's done, now our job is to find out how much you know," says Johnson. They log on to a website called TaskStream and pluck the first assignment they see. The institution promises that every assignment will be graded within two days of submission.</p> <p>Emily L. Child, one of the evaluators, is a stay-at-home mother of three who lives near Salt Lake City. She grades 10 to 15 assignments per day, six days a week. She estimates that she has graded 14,400 assignments in the six years she has worked for the university.</p> <p>Western Governors requires all evaluators to hold at least a master's degree in the subject they're grading. Child, a former teacher, grades assignments only in the education major. A typical assignment, she says, involves a student's submitting a sample lesson plan or classroom strategies.</p> <p>Evaluators write extensive comments on each task, explaining why the student passed or failed to prove competence in the requisite skill. No letter grades are given. Officials say a pass in a Western Governors course amounts to a B at a traditional university.</p> <p>All evaluators receive a month of training, conducted online, about how to follow each task's grading guidelines, which lay out characteristics of a passing score.</p> <p>The identities of the evaluators are hidden from students and mentors. The goal is to protect the graders from students nagging them about grades, or from mentors who might lobby to pass a borderline student to better reflect on their teaching.</p> <p>The graders regularly participate in "calibration exercises," in which they grade a simulated assignment to make sure they are all scoring consistently. The process is designed to run like a well-oiled machine.</p> <p>Some evaluators objected to the system at first, says Johnson, especially professors who come from traditional higher education. Some insist that they don't need to justify each grade they give, arguing that they know a passing assignment when they see it. "That's hogwash," she says. "If you know it when you see it, then tell us what it is you see."</p> <p>Other evaluators want to push talented students to do more than the university's requirements, or allow a struggling student to pass if he or she is just under the bar. "Some people just can't acclimate to a competency-based environment," says Johnson. "I tell them, if they don't buy this, they need to not be here."</p> <p>Even Johnson had to be convinced when she started at Western Governors, after having taught school and helped to develop instructional standards for the Utah State Office of Education. "I was an academic snob," she says. "As I was going through their training, 1 began to think, Oh, my gosh I think they have something here."</p> <p>Besides aiming for validity in grading, the Western Governors infrastructure is designed to stretch or compress like an accordion: part-time graders can be added if enrollment spikes, or graders can work fewer hours if enrollment drops. In the past two years, the student population has grown from 14,200 to more than 25,000, and the evaluators have graded more than 1.1 million assignments.</p> <hd id="AN0067664250-3">Meet the Robots</hd> <p>Pam Thomas, an instructor of biology at the University of Central Florida, decided to try robot grading because she loves to teach large classes. She had more than 1,000 in her "General Biology" course last spring, and she wanted to give them assignments more challenging than "punching buttons on multiple-choice tests."</p> <p>When she announced to her class that software would automatically grade essay tests, many students were wary. "The students said, I'm being graded by a robot?" she remembers. "I said, Anybody who doesn't get a 100, 1 will look at a machine, and I will see if the machine made a mistake."</p> <p>Some students did challenge their scores, but in most cases the computer proved correct.</p> <p>Then Thomas performed an experiment. She and some teaching assistants scored the tests by hand and compared their performance with the computer's. "The TA's were way far off on inter-rater reliability," she says, referring to whether different people scored the same test consistently. The graduate students became fatigued and made mistakes after grading several tests in a row, she told me, "but the machine was right-on every time."</p> <p>Setting up the software takes some doing. Thomas had to carefully lay out what constituted a correct response, so the machine knew what to look for.</p> <p>1 called up Idea Works, the company that makes the software, SAGrader. The program is a tough sell, even after six years on the market. "People are a little skeptical that what we're doing is possible," says Colin Monaghan, product manager for SAGrader. Only a couple of colleges are testing it, including Park University and the University of Missouri at Columbia.</p> <p>The program was developed by a Missouri professor, Edward E. Brent, who has a joint appointment in the sociology and computer-science departments. He argues that students like the idea that tests are evaluated in a consistent way. "We have a number of courses at my institution that have multiple sections of the same course, and differing graders are grading the same assignment," he says. "With the SAGrader program, the students can be assured that it's consistent across sections."</p> <p>Another selling point is the software's fast response rate. It can grade 1,000 essay tests in minutes. Professors can set the software to return the grade immediately and can give students the option of making revisions and resubmitting their work on the spot.</p> <p>"Usually in school you turn something in and you never see it for two weeks," says Monaghan. Once students get essays back instantly, they start to view essay tests differently. "It's almost like a big math problem. You don't expect to get everything right the first time, but you work through it."</p> <p>Even if adoption is sluggish, robot grading is the hottest trend in testing circles, says Jacqueline Leighton, a professor of educational psychology at the University of Alberta who edits the journal Educational Measurement: Issues and Practice. Companies building essay-grading robots include the Educational Testing Service, which sells e-rater, and Pearson Education, which makes Intelligent Essay Assessor. "The research is promising, but they're still very much in their infancy," Leighton says.</p> <p>Skepticism is still the most common response, though.</p> <p>"I cannot imagine outsourcing the grading," says Angela Linse, executive director of the Schreyer Institute for Teaching Excellence, at Pennsylvania State University. "The public already thinks that faculty have cushy jobs. I cannot imagine what would happen if we outsourced grading."</p> <p>She is among those who feel that grade inflation is not as bad as critics describe it. "From my experience, faculty are very diligent about grading fairly and consistently," she says. And she argues that most professors don't give out high grades just to win positive student evaluations.</p> <p>Linse does encourage professors to take some advice given to the graders at Western Governors: Try to write out rubrics for assignments and show them to students, to be clear about what is expected.</p> <p>Researchers found that average grades have risen for 30 years, and that A is now the most common grade given at most colleges.</p> <p>The process is designed to run like a well-oiled machine.</p> <p>Condensed, with permission, from The Chronicle of Higher Education, 57 (August 12, 2011), A1, 4-5. Copyright 2011, The Chronicle of Higher Education. The complete version of the article is available at <ulink href="http://chronicle.com">http://chronicle.com</ulink>.</p> <aug> <p>By JEFFREY R. YOUNG</p> <p></p> <p>Jeffrey R. Young is a senior writer for The Chronicle of Higher Education.</p> </aug> |
|---|---|
| Header | DbId: eric DbLabel: ERIC An: EJ964247 AccessLevel: 3 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Professors Cede Grading Power to Outsiders--Even Computers – Name: Language Label: Language Group: Lang Data: English – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Young%2C+Jeffrey+R%2E%22">Young, Jeffrey R.</searchLink> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="SO" term="%22Education+Digest%3A+Essential+Readings+Condensed+for+Quick+Review%22"><i>Education Digest: Essential Readings Condensed for Quick Review</i></searchLink>. Dec 2011 77(4):24-28. – Name: Avail Label: Availability Group: Avail Data: Prakken Publications. 832 Phoenix Drive, P.O. Box 8623, Ann Arbor, MI 48108. Tel: 734-975-2800; Fax: 734-975-2787; Web site: http://www.eddigest.com/ – Name: PeerReviewed Label: Peer Reviewed Group: SrcInfo Data: N – Name: Pages Label: Page Count Group: Src Data: 5 – Name: DatePubCY Label: Publication Date Group: Date Data: 2011 – Name: TypeDocument Label: Document Type Group: TypDoc Data: Journal Articles<br />Reports - Descriptive – Name: Audience Label: Education Level Group: Audnce Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink> – Name: Subject Label: Descriptors Group: Su Data: <searchLink fieldCode="DE" term="%22Grading%22">Grading</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluators%22">Evaluators</searchLink><br /><searchLink fieldCode="DE" term="%22Outsourcing%22">Outsourcing</searchLink><br /><searchLink fieldCode="DE" term="%22Computers%22">Computers</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Scoring+Machines%22">Test Scoring Machines</searchLink><br /><searchLink fieldCode="DE" term="%22Virtual+Universities%22">Virtual Universities</searchLink><br /><searchLink fieldCode="DE" term="%22Public+Colleges%22">Public Colleges</searchLink> – Name: Subject Label: Geographic Terms Group: Su Data: <searchLink fieldCode="DE" term="%22Florida%22">Florida</searchLink> – Name: ISSN Label: ISSN Group: ISSN Data: 0013-127X – Name: Abstract Label: Abstract Group: Ab Data: The best way to eliminate grade inflation is to take professors out of the grading process: Replace them with professional evaluators who never meet students and don't worry that students will punish harsh grades with poor reviews. That's the argument made by leaders of Western Governors University, which has hired 300 adjunct professors who do nothing but grade student work. Western Governors is not the only institution reassessing grading. A few others, including the University of Central Florida, now outsource scoring of some essay tests to computers. Their software can grade essays thanks to improvements in artificial-intelligence techniques. Software has no emotional biases, either, and one instructor says machines have proved more fair and balanced in grading than humans. Naturally, the standard grading model has plenty of defenders, including some who argue that claims of grade inflation are exaggerated. The current system forges a nurturing relationship between instructor and student and gives individualized attention that no robot or stranger could give, this argument goes. But the efforts at Western Governors and Central Florida could change that relationship, and point to ways to pop any grade-inflation bubble. – Name: AbstractInfo Label: Abstractor Group: Ab Data: ERIC – Name: DateEntry Label: Entry Date Group: Date Data: 2012 – Name: URL Label: Access URL Group: URL Data: <link linkTarget="URL" linkTerm="https://www.eddigest.com/index.php" linkWindow="_blank">http://www.eddigest.com/index.php</link> – Name: AN Label: Accession Number Group: ID Data: EJ964247 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ964247 |
| RecordInfo | BibRecord: BibEntity: Languages: – Text: English PhysicalDescription: Pagination: PageCount: 5 StartPage: 24 Subjects: – SubjectFull: Grading Type: general – SubjectFull: Evaluators Type: general – SubjectFull: Outsourcing Type: general – SubjectFull: Computers Type: general – SubjectFull: Test Scoring Machines Type: general – SubjectFull: Virtual Universities Type: general – SubjectFull: Public Colleges Type: general – SubjectFull: Florida Type: general Titles: – TitleFull: Professors Cede Grading Power to Outsiders--Even Computers Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Young, Jeffrey R. IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 12 Type: published Y: 2011 Identifiers: – Type: issn-print Value: 0013-127X Numbering: – Type: volume Value: 77 – Type: issue Value: 4 Titles: – TitleFull: Education Digest: Essential Readings Condensed for Quick Review Type: main |
| ResultId | 1 |
