How We Learn to Make Decisions: Rapid Propagation of Reinforcement Learning Prediction Errors in Humans.
Saved in:
| Title: | How We Learn to Make Decisions: Rapid Propagation of Reinforcement Learning Prediction Errors in Humans. |
|---|---|
| Authors: | Krigolson, Olav E., Hassall, Cameron D., Handy, Todd C. |
| Source: | Journal of Cognitive Neuroscience. 2014, Vol. 26 Issue 3, p635-644. 10p. 4 Graphs. |
| Subjects: | Brain imaging, Reinforcement learning, Psychological feedback, Mesencephalon, Neurosciences, Decision making, Dopamine |
| Abstract: | Our ability to make decisions is predicated upon our knowledge of the outcomes of the actions available to us. Reinforcement learning theory posits that actions followed by a reward or punishment acquire value through the computation of prediction errors--discrepancies between the predicted and the actual reward. A multitude of neuroimaging studies have demonstrated that rewards and punishments evoke neural responses that appear to reflect reinforcement learning prediction errors [e.g., Krigolson, O. E., Pierce, L.J., Holroyd, C. B., & Tanaka, J. W. Learning to become an expert: Reinforcement learning and the acquisition of perceptual expertise. Journal of Cognitive Neuroscience, 21, 1833-1840, 2009; Bayer, H. M., & Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47, 129-141, 2005; O'Doherty, J. P. Reward representations and reward-related learning in the human brain: Insights from neuroimaging. Current Opinion in Neurobiology, 14, 769-776, 2004; Holroyd, C. B., & Coles, M. G. H. The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679-709, 2002]. Here, we used the brain ERP technique to demonstrate that not only do rewards elicit a neural response akin to a prediction error but also that this signal rapidly diminished and propagated to the time of choice presentation with learning. Specifically, in a simple, learnable gambling task, we show that novel rewards elicited a feedback error-related negativity that rapidly decreased in amplitude with learning. Furthermore, we demonstrate the existence of a reward positivity at choice presentation, a previously unreported ERP component that has a similar timing and topography as the feedback error-related negativity that increased in amplitude with learning. The pattern of results we observed mirrored the output of a computational model that we implemented to compute reward prediction errors and the changes in amplitude of these prediction errors at the time of choice presentation and reward delivery. Our results provide further support that the computations that underlie human learning and decision-making follow reinforcement learning principles. [ABSTRACT FROM AUTHOR] |
| Copyright of Journal of Cognitive Neuroscience is the property of MIT Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Psychology and Behavioral Sciences Collection |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: pbh DbLabel: Psychology and Behavioral Sciences Collection An: 94263063 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: How We Learn to Make Decisions: Rapid Propagation of Reinforcement Learning Prediction Errors in Humans. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Krigolson%2C+Olav+E%2E%22">Krigolson, Olav E.</searchLink><br /><searchLink fieldCode="AR" term="%22Hassall%2C+Cameron+D%2E%22">Hassall, Cameron D.</searchLink><br /><searchLink fieldCode="AR" term="%22Handy%2C+Todd+C%2E%22">Handy, Todd C.</searchLink> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Journal+of+Cognitive+Neuroscience%22">Journal of Cognitive Neuroscience</searchLink>. 2014, Vol. 26 Issue 3, p635-644. 10p. 4 Graphs. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Brain+imaging%22">Brain imaging</searchLink><br /><searchLink fieldCode="DE" term="%22Reinforcement+learning%22">Reinforcement learning</searchLink><br /><searchLink fieldCode="DE" term="%22Psychological+feedback%22">Psychological feedback</searchLink><br /><searchLink fieldCode="DE" term="%22Mesencephalon%22">Mesencephalon</searchLink><br /><searchLink fieldCode="DE" term="%22Neurosciences%22">Neurosciences</searchLink><br /><searchLink fieldCode="DE" term="%22Decision+making%22">Decision making</searchLink><br /><searchLink fieldCode="DE" term="%22Dopamine%22">Dopamine</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Our ability to make decisions is predicated upon our knowledge of the outcomes of the actions available to us. Reinforcement learning theory posits that actions followed by a reward or punishment acquire value through the computation of prediction errors--discrepancies between the predicted and the actual reward. A multitude of neuroimaging studies have demonstrated that rewards and punishments evoke neural responses that appear to reflect reinforcement learning prediction errors [e.g., Krigolson, O. E., Pierce, L.J., Holroyd, C. B., & Tanaka, J. W. Learning to become an expert: Reinforcement learning and the acquisition of perceptual expertise. Journal of Cognitive Neuroscience, 21, 1833-1840, 2009; Bayer, H. M., & Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47, 129-141, 2005; O'Doherty, J. P. Reward representations and reward-related learning in the human brain: Insights from neuroimaging. Current Opinion in Neurobiology, 14, 769-776, 2004; Holroyd, C. B., & Coles, M. G. H. The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679-709, 2002]. Here, we used the brain ERP technique to demonstrate that not only do rewards elicit a neural response akin to a prediction error but also that this signal rapidly diminished and propagated to the time of choice presentation with learning. Specifically, in a simple, learnable gambling task, we show that novel rewards elicited a feedback error-related negativity that rapidly decreased in amplitude with learning. Furthermore, we demonstrate the existence of a reward positivity at choice presentation, a previously unreported ERP component that has a similar timing and topography as the feedback error-related negativity that increased in amplitude with learning. The pattern of results we observed mirrored the output of a computational model that we implemented to compute reward prediction errors and the changes in amplitude of these prediction errors at the time of choice presentation and reward delivery. Our results provide further support that the computations that underlie human learning and decision-making follow reinforcement learning principles. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Journal of Cognitive Neuroscience is the property of MIT Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=pbh&AN=94263063 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1162/jocn_a_00509 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 10 StartPage: 635 Subjects: – SubjectFull: Brain imaging Type: general – SubjectFull: Reinforcement learning Type: general – SubjectFull: Psychological feedback Type: general – SubjectFull: Mesencephalon Type: general – SubjectFull: Neurosciences Type: general – SubjectFull: Decision making Type: general – SubjectFull: Dopamine Type: general Titles: – TitleFull: How We Learn to Make Decisions: Rapid Propagation of Reinforcement Learning Prediction Errors in Humans. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Krigolson, Olav E. – PersonEntity: Name: NameFull: Hassall, Cameron D. – PersonEntity: Name: NameFull: Handy, Todd C. IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 03 Text: 2014 Type: published Y: 2014 Identifiers: – Type: issn-print Value: 0898929X Numbering: – Type: volume Value: 26 – Type: issue Value: 3 Titles: – TitleFull: Journal of Cognitive Neuroscience Type: main |
| ResultId | 1 |
