Improving Metacognition: A Comparison of Interventions
Saved in:
| Title: | Improving Metacognition: A Comparison of Interventions |
|---|---|
| Language: | English |
| Authors: | Saenz, Gabriel D. (ORCID |
| Source: | Applied Cognitive Psychology. Sep-Oct 2019 33(5):918-929. |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 12 |
| Publication Date: | 2019 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Metacognition, Intervention, Prediction, Accuracy, Feedback (Response), Motivation, Incentives, Reflection |
| DOI: | 10.1002/acp.3556 |
| ISSN: | 0888-4080 |
| Abstract: | Accurate knowledge monitoring is critical to the learning process, as it allows one to regulate studying and test preparation. Thus, a number of investigations have attempted to improve metacognition in the classroom, with the ultimate goal of improving student exam performance. However, such interventions have had inconsistent success using varying paradigms. We compared the effectiveness of five interventions aimed at improving prediction accuracy in a laboratory environment: review, salient feedback, motivation warning lecture, incentives, and reflection. Only the salient feedback and the motivation warning lecture interventions significantly improved participants' prediction accuracy from test 1 to test 2. Review, incentives, and reflection did not improve predictive or postdictive calibration. Well-timed salient feedback and a lecture warning students not to be biased by desired grades were effective methods of improving calibration accuracy. Results offer effective interventions to improve metacognition that could be used in a classroom setting. |
| Abstractor: | As Provided |
| Entry Date: | 2020 |
| Accession Number: | EJ1262144 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFTEBAd7D6txD3uNJibB3JWAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDDVy_iwLbYqtaVo3nwIBEICBmhd2pxEQFmkbyz8eaBv2w9EixeuKH1ZtmVKyAv3yi59AzlgKPqp3yKsXyqP0Z80Xiu8MGEPQQrm85Qlr8ALwaiNdODmGsMUHuHQUWSp5vZMC3avmy7K3JPLYTRhCsnkcSzrtvzVI8kkaF5He53kH_tJkHNMFJKK5khd1WcKTaZebyKQMlIjwPPnXC9VIT-3y6AJv7q1EICUAe_A= Text: Availability: 1 Value: <anid>AN0138393115;bu801sep.19;2019Sep04.03:28;v2.2.500</anid> <title id="AN0138393115-1">Improving metacognition: A comparison of interventions </title> <p>Summary Accurate knowledge monitoring is critical to the learning process, as it allows one to regulate studying and test preparation. Thus, a number of investigations have attempted to improve metacognition in the classroom, with the ultimate goal of improving student exam performance. However, such interventions have had inconsistent success using varying paradigms. We compared the effectiveness of five interventions aimed at improving prediction accuracy in a laboratory environment: review, salient feedback, motivation warning lecture, incentives, and reflection. Only the salient feedback and the motivation warning lecture interventions significantly improved participants' prediction accuracy from test 1 to test 2. Review, incentives, and reflection did not improve predictive or postdictive calibration. Well‐timed salient feedback and a lecture warning students not to be biased by desired grades were effective methods of improving calibration accuracy. Results offer effective interventions to improve metacognition that could be used in a classroom setting.</p> <p>Keywords: calibration; classroom; interventions; metacognition; postdictions; predictions</p> <hd id="AN0138393115-2">INTRODUCTION</hd> <p>Educators sometimes observe a disconnect between students' expectations and their performance, as students often seem surprised by poor performance. A large number of studies corroborate this experience showing that students' performance predictions are often inaccurate, usually in the direction of overconfidence (Bol, Hacker, O'Shea, &amp; Allen, [<reflink idref="bib2" id="ref1">2</reflink>]; Dunning, Johnson, Ehrlinger, &amp; Kruger, [<reflink idref="bib6" id="ref2">6</reflink>]; Ehrlinger, Johnson, Banner, Dunning, &amp; Kruger, [<reflink idref="bib7" id="ref3">7</reflink>]; Hacker, Bol, Horgan, &amp; Rakow, [<reflink idref="bib15" id="ref4">15</reflink>]; Helzer &amp; Dunning, [<reflink idref="bib18" id="ref5">18</reflink>]; Miller &amp; Geraci, [<reflink idref="bib24" id="ref6">24</reflink>]; Nietfeld, Cao, &amp; Osborne, [<reflink idref="bib28" id="ref7">28</reflink>]; Saenz, Geraci, Miller, &amp; Tirso, [<reflink idref="bib32" id="ref8">32</reflink>]). Further, these studies consistently show that the lowest performing students are the most inaccurate, suggesting a link between poor performance and poor metacognitive accuracy. These individuals are said to be "unskilled and unaware" (Kruger &amp; Dunning, [<reflink idref="bib21" id="ref9">21</reflink>]) because they lack test knowledge and are unaware of this fact. The finding that low performers overpredict their performance has been interpreted causally. Because they have not learned the test material, they are unaware that they are missing key knowledge, which impedes them from choosing to study and attain the knowledge.</p> <p>There is extensive evidence for a link between metacognition and performance from both classroom (Bol et al., [<reflink idref="bib2" id="ref10">2</reflink>]; Dunning et al., [<reflink idref="bib6" id="ref11">6</reflink>]; Ehrlinger et al., [<reflink idref="bib7" id="ref12">7</reflink>]; Foster, Was, Dunlosky, &amp; Isaacson, [<reflink idref="bib10" id="ref13">10</reflink>]; Hacker et al., [<reflink idref="bib15" id="ref14">15</reflink>]; Helzer &amp; Dunning, [<reflink idref="bib18" id="ref15">18</reflink>]; Miller &amp; Geraci, [<reflink idref="bib24" id="ref16">24</reflink>]; Nietfeld et al., [<reflink idref="bib28" id="ref17">28</reflink>]; Saenz et al., [<reflink idref="bib32" id="ref18">32</reflink>]) and laboratory studies (Dunlosky &amp; Rawson, [<reflink idref="bib4" id="ref19">4</reflink>]; Hartwig &amp; Dunlosky, [<reflink idref="bib17" id="ref20">17</reflink>]; Kruger &amp; Dunning, [<reflink idref="bib21" id="ref21">21</reflink>]; Miller &amp; Geraci, [<reflink idref="bib25" id="ref22">25</reflink>], [<reflink idref="bib26" id="ref23">26</reflink>], [<reflink idref="bib27" id="ref24">27</reflink>]). Further, metacognitive ability is correlated with overall academic performance (Everson &amp; Tobias, [<reflink idref="bib8" id="ref25">8</reflink>]; Kelemen, Winningham, &amp; Weaver, [<reflink idref="bib20" id="ref26">20</reflink>]; Thiede, [<reflink idref="bib36" id="ref27">36</reflink>]; Thiede, Anderson, &amp; Therriault, [<reflink idref="bib37" id="ref28">37</reflink>]). In light of these findings, a number of interventions have been designed to improve the accuracy of students' metacognition, with the ultimate goal of improving test performance (see Hacker, Bol, &amp; Keener, [<reflink idref="bib16" id="ref29">16</reflink>], for review).</p> <p>Many interventions have used some form of feedback in an attempt to improve students' calibration (e.g., Foster et al., [<reflink idref="bib10" id="ref30">10</reflink>]; Hacker et al., [<reflink idref="bib15" id="ref31">15</reflink>]; Nietfeld et al., [<reflink idref="bib28" id="ref32">28</reflink>]; Nietfeld, Cao, &amp; Osborne, [<reflink idref="bib29" id="ref33">29</reflink>]). Students are given feedback about their performance, their predictions, or both so that they may "calibrate" subsequent predictions and performance to each other. Often, feedback consists of simply presenting students with their test grades in hopes that such information will allow students to adjust and improve their prediction accuracy (typically lowering their predictions to match their performance). Such a methodology may mirror the type of feedback students normally attain in a classroom environment.</p> <p>Evidence for the effectiveness of feedback for calibration improvement is mixed. For example, simply making prediction and performance information available is not sufficient to produce calibration change. Foster et al. ([<reflink idref="bib10" id="ref34">10</reflink>]) used a feedback‐only intervention by exposing students to 13 testing–feedback cycles in one semester. Yet even after 13 cycles of feedback, there was no improvement in calibration (referred to as "bias" in their study). Further, the amount of feedback does not seem to be strongly associated with improvement in calibration, a conclusion suggested by Hacker, Bol, and Keener ([<reflink idref="bib16" id="ref35">16</reflink>]) and affirmed by Foster et al. ([<reflink idref="bib10" id="ref36">10</reflink>]). Although extensive feedback alone may not lead to calibration improvement, feedback cannot be ruled out as a potentially effective intervention. There is evidence that the "strength" of feedback (and not the amount) is key for calibration improvement (Hacker, Bol, &amp; Keener, [<reflink idref="bib16" id="ref37">16</reflink>]). For example, whereas passive, self‐directed feedback may not produce any improvement in calibration (Nietfeld et al., [<reflink idref="bib28" id="ref38">28</reflink>]), having students work through education‐monitoring exercises, combined with guided feedback, results in improved calibration over the course of the semester (Nietfeld et al., [<reflink idref="bib29" id="ref39">29</reflink>]). Thus, interventions that combine test practice with feedback seem to improve metacognitive accuracy. It is also possible that sufficiently "strong" feedback alone could improve metacognition.</p> <p>Although feedback interventions are common, feedback may not be required to improve prediction accuracy. The basic assumption of a feedback intervention is that people are unable to make accurate performance predictions because they do not have enough information, about themselves, the exam, others (in the case of relative judgments), or all three. However, a lack of information is not the only possible source of prediction errors. For example, performance predictions may be associated with people's motivations (Helzer &amp; Dunning, [<reflink idref="bib18" id="ref40">18</reflink>]; Saenz et al., [<reflink idref="bib32" id="ref41">32</reflink>]; Serra &amp; DeMarree, [<reflink idref="bib33" id="ref42">33</reflink>]), and interventions designed to reduce students' reliance on their motivations when making predictions can be effective (Saenz et al., [<reflink idref="bib32" id="ref43">32</reflink>]). In the Saenz et al. study, students received feedback about the accuracy of their predictions and were warned about relying on their desired grades to make predictions, which tends to lead to errors. Before their next exam, students were again warned about the potentially biasing nature of grade desires and told to rely instead on academic information (e.g., amount of studying) as they predicted their grades for the upcoming test. Results showed that the motivation warning lecture plus feedback produced improvement in calibration and test performance. Further, academic information was more associated and grade desires were less associated with predictions following the intervention. Thus, there is evidence that interventions that target students' motivations may be effective.</p> <p>Other interventions offer incentives (e.g., money and bonus points) for accurate predictions. These interventions rely on the assumption that people make inaccurate predictions because they are not sufficiently motivated to thoughtfully evaluate their knowledge but that with the right incentive, they can be more metacognitively accurate. Of course, incentive interventions do not provide students with any additional knowledge about themselves or the test, providing no clear avenue for accuracy improvement. Perhaps not surprisingly then, incentive interventions have had mixed results. Incentives have led to improvement in calibration when used to supplement other types of interventions (Callender, Franco‐Watkins, &amp; Roberts, [<reflink idref="bib3" id="ref44">3</reflink>]; Gutierrez &amp; Schraw, [<reflink idref="bib13" id="ref45">13</reflink>]; Hacker, Bol, &amp; Bahbahani, [<reflink idref="bib14" id="ref46">14</reflink>]; Hogarth, Gibbs, McKenzie, &amp; Marquis, [<reflink idref="bib19" id="ref47">19</reflink>]; Miller &amp; Geraci, [<reflink idref="bib24" id="ref48">24</reflink>]). However, incentive‐only interventions have yielded no improvement in calibration (Ehrlinger et al., [<reflink idref="bib7" id="ref49">7</reflink>]).</p> <p>The use of practice tests is another intervention designed to improve students' predictive accuracy. Practice tests may improve metacognition because they are essentially a form of performance feedback (Dunlosky, Rawson, &amp; McDonald, [<reflink idref="bib5" id="ref50">5</reflink>]; Snooks, [<reflink idref="bib34" id="ref51">34</reflink>]). However, practice tests are only sometimes effective at improving students' calibrations. Although a brief but difficult practice test can improve predictive accuracy (Miller &amp; Geraci, [<reflink idref="bib26" id="ref52">26</reflink>]), other practice test interventions decrease predictive accuracy (Bol &amp; Hacker, [<reflink idref="bib1" id="ref53">1</reflink>]).</p> <p>We were also interested in whether people could improve their calibration accuracy if given enough time to carefully consider their predictions. Researchers have not explored the amount of time associated with making grade predictions. Although it may take a relatively short period of time to make a grade prediction, it could take longer to make an accurate grade prediction. Many metacognitive studies do not specify the length of time dedicated to their metacognitive judgments (Foster et al., [<reflink idref="bib10" id="ref54">10</reflink>]; Miller &amp; Geraci, [<reflink idref="bib24" id="ref55">24</reflink>], [<reflink idref="bib25" id="ref56">25</reflink>]; Nietfeld et al., [<reflink idref="bib28" id="ref57">28</reflink>], [<reflink idref="bib29" id="ref58">29</reflink>]). We speculate that if people are motivated to make overconfident predictions (i.e., Saenz et al., [<reflink idref="bib32" id="ref59">32</reflink>]), then it may take some time for them to override any predisposition to be overconfident. A reflection intervention was designed to test this possibility by having participants spend time making their grade predictions.</p> <p>Although researchers have devoted much time and energy towards testing various interventions, how individual interventions compare with one another remains unknown. To answer this question, we compared the effectiveness of five different interventions in a controlled laboratory environment: review (including test practice and self‐generated feedback), salient feedback, a motivation warning lecture, incentives, and reflection. We designed the interventions with an eye towards external validity, meaning that they were relatively short (10 min) and could be used in a classroom environment.</p> <hd id="AN0138393115-3">METHOD</hd> <p></p> <hd id="AN0138393115-4">Participants</hd> <p>A total of 223 undergraduate psychology students from Texas A&amp;M University participated in return for partial course credit. Five participants were removed from the data analyses for failing to follow directions, resulting in a total of 218 participants. Participants were run in groups of 1–10 at a time. Intervention condition and test order were both counterbalanced across participants.</p> <hd id="AN0138393115-5">Design</hd> <p>We used a 6 × 2 mixed design, in which Intervention Condition (five interventions vs. control) served as the between‐subjects variable, and Prediction Time (before or after the intervention) was the within‐subjects variable. The dependent variable was Calibration (the absolute difference between prediction and performance) change from test 1 to 2. Calibration change was measured in two ways: from prediction 1 to prediction 2 and from postdiction 1 to postdiction 2.</p> <hd id="AN0138393115-6">Procedure</hd> <p>Participants took two logical reasoning exams composed of different questions from the same question bank. Participants were briefly informed about the content and origins of the test before their initial predictions. Before the exam, they were asked to predict (prediction 1) how they would perform on the test using a 0–100% scale ("What grade do you think you will receive on this test? ___%"). Immediately after each test, participants were asked to make a postdiction (postdiction 1) on a 0–100% scale ("What grade do you think you will receive on this test? ___%"). Participants were then given one of five 10‐min interventions, or they completed puzzles for 10 min in the control condition, before completing another cycle of testing and judgments (prediction 2, postdiction 2).</p> <hd id="AN0138393115-7">Materials</hd> <p></p> <hd id="AN0138393115-8">Logical reasoning test</hd> <p>Participants completed two logical reasoning tests taken from the Official Law School Admission Test PrepTest of June 2007 and Law School Admission Test self‐assessment modules (testprepreview.com; see Appendix A for sample questions). The tests were designed to be very difficult while avoiding floor effects to allow room for calibration improvements. As such, average performance was low (<emph>M</emph> = 37.11%, <emph>SD</emph> = 13.46). Each test consisted of 20 multiple‐choice questions to be answered in 20 min. Test order was counterbalanced between subjects.</p> <hd id="AN0138393115-9">Interventions</hd> <p>All interventions and control conditions were designed to take approximately 10 min.</p> <hd id="AN0138393115-10">Review intervention</hd> <p>This condition included two forms of review: reviewing representative test questions and reviewing performance predictions. Without being told their performance or prediction accuracy on the previously completed test, participants in this intervention condition had an opportunity to observe their test performance and prediction accuracy on similar test questions before their second test. This intervention mimicked the classroom practice of giving students review questions before an exam so they may gauge their preparedness. Participants practiced four logical reasoning questions from the same question pool as the ones used to make the logical reasoning exams. Review items were chosen to mimic the average level of difficulty on the two critical tests. Participants heard each question read aloud and were given a minute to answer the question, as well as to predict whether they thought their answer was correct ("I think I answered this question correctly: yes/no"). Participants were shown and told the answer and given a short description explaining the answer. Participants were then given 1 min to review their results and respond to a brief self‐monitoring accuracy survey (see Appendix B for a sample of questions, feedback, and survey items used). Instead of providing students with direct feedback, the review condition was designed so that participants could produce their own feedback by responding to survey questions about the accuracy of their prior responses.</p> <hd id="AN0138393115-11">Saliency intervention</hd> <p>Participants were given salient feedback on their performance and prediction accuracy from the first test. After taking the first test, the experimenter graded the exams and gave each participant a feedback form that included the participant's test grade, prediction, and calibration in large font. Participants were then given a verbal explanation of calibration and told to think about the feedback for 5 min (grading took approximately 5 min, totaling 10 min). Participants were informed of the formula and meaning of their calibration number and told that a more positive calibration score indicated greater overconfidence whereas a more negative calibration score indicated greater underconfidence. To prevent underpreparation in a classroom context, participants were further told that either perfect accuracy or underconfidence may be more desirable than overconfidence. To enhance feedback saliency, the forms remained with the participant during the remainder of the study (see Appendix B for a sample feedback form).</p> <hd id="AN0138393115-12">Motivation warning lecture intervention</hd> <p>This intervention was designed to reduce students' reliance on motivations that might affect their grade predictions and to encourage the use of academic information that should lead to more accurate predictions (Helzer &amp; Dunning, [<reflink idref="bib18" id="ref60">18</reflink>]; Saenz et al., [<reflink idref="bib32" id="ref61">32</reflink>]; Serra &amp; DeMarree, [<reflink idref="bib33" id="ref62">33</reflink>]). Participants watched a 10‐min prerecorded video lecture adapted from the motivation warning lecture used in Saenz et al. ([<reflink idref="bib32" id="ref63">32</reflink>]). Participants learned about grade predictions and calibration and that students (particularly low performers) are often overconfident. Students were then told that grade predictions can be affected by motivational factors (such as how well people would like to perform) and were encouraged to avoid these motivational biases. Instead, students were encouraged to rely on performance or familiarity with test material to inform their grade predictions. Original materials for this intervention may be accessed at https://osf.io/shjad/?view_only=0faf6b5f43a848a596b5e35cacad3ac9</p> <hd id="AN0138393115-13">Incentive intervention</hd> <p>Participants were given the opportunity to earn $50 for accurate predictions. Participants received a lengthy (6‐min) explanation instructing that the participant with the most accurate grade prediction following the second exam would receive the reward. This explanation was designed to make the reward very salient and mentioned the "50 dollar prize" repeatedly throughout, while still requiring participants to put forth a reasonable effort during the tests (i.e., answering no questions and predicting a 0 was not rewarded). Participants were informed that one in about 40 participants would receive the reward. Participants then provided their contact information to be notified if they won (about 4 min). These procedures were designed to make the incentive as salient as possible (see Appendix B for a sample ballot).</p> <hd id="AN0138393115-14">Reflection intervention</hd> <p>This intervention was designed to give participants the opportunity to think about their predictions for an extended period of time, without receiving any feedback. Participants made a prediction once a minute for the 10‐min intervention period, resulting in 10 predictions in total (see Appendix B for a sample form). Unlike the other interventions, this one does not necessarily represent what an instructor might do but instead what students may do on their own accord. It is conceivable that students who are concerned about their future test performance may consider their knowledge for an extended period of time. This may take the form of assessing their knowledge, contemplating for a moment, and then making a new assessment of their knowledge within a short period of time, perhaps multiple times; the reflection intervention was designed to capture this process.</p> <hd id="AN0138393115-15">Control</hd> <p>Participants in the control condition completed maze puzzles for 10 min.</p> <hd id="AN0138393115-16">RESULTS</hd> <p></p> <hd id="AN0138393115-17">Exam performance</hd> <p>We ran a repeated‐measures analysis of variance (ANOVA) comparing test performance between the first and second exams and between the six intervention conditions. There was no interaction between Intervention Condition and test order on logical reasoning test performance, <emph>F</emph>(<reflink idref="bib1" id="ref64">1</reflink>, 216) = 0.22, <emph>p</emph> = .641, partial <emph>η</emph><sups>2</sups> &lt; .01, and there was no main effect of Intervention Condition on performance, <emph>F</emph>(<reflink idref="bib1" id="ref65">1</reflink>, 216) = 2.19, <emph>p</emph> = .141, partial <emph>η</emph><sups>2</sups> = .01. There was a significant main effect of test order on performance, <emph>F</emph>(<reflink idref="bib1" id="ref66">1</reflink>, 216) = 7.60, <emph>p</emph> = .006, partial <emph>η</emph><sups>2</sups> = .03, wherein participants performed better on the first test (<emph>M</emph> = 38.98, <emph>SD</emph> = 15.91) than on the second (<emph>M</emph> = 35.87, <emph>SD</emph> = 15.27). These results may be attributable to testing fatigue and amounted to less than one question's difference in performance (5%). Nevertheless, this small decrease in test score was reflected in lowered judgment accuracy during the second testing cycle in the control condition. Therefore, the control condition was used as a baseline to control for nonintervention‐associated changes in test performance and calibration. The two tests were not very reliable within themselves (Cronbach's <emph>α</emph> = .594 and.547), but scores between the first and second tests were significantly correlated (<emph>r</emph> = .422, <emph>p</emph> &lt; .001).</p> <hd id="AN0138393115-18">Calibration</hd> <p>To examine the effects of the interventions on calibration, we first calculated prediction accuracy using absolute centered calibration (ACC) scores. ACC scores were calculated by subtracting predictions from performance, taking the absolute value of the resulting difference score, and centering these absolute calibration scores around the control condition by subtracting the control condition mean scores from each other condition.</p> <p>In the control condition, grade predictions dropped from prediction 1 (<emph>M</emph> = 79.92, <emph>SD</emph> = 8.79) to postdiction 1 (<emph>M</emph> = 60.85, <emph>SD</emph> = 16.17) but remained stable across the remaining judgments (prediction 2 and postdiction 2). Note that the change in test performance and lack of change in control condition predictions produced the aforementioned drop in accuracy from test 1 to test 2 (Table ). The centering of absolute calibration scores should reduce the impact of nonintervention‐related calibration change, such that interventions had to produce calibration change over and above that found in the control condition to be considered significant. When interpreting ACC scores, larger numbers indicate greater inaccuracy than did control, and smaller numbers indicate greater accuracy than did control. ACC scores are centered, so 0 represents results equivalent to the control condition, and a negative ACC score indicates that predictions were more accurate than control (Table ). Table  and Figures  and report ACC improvement, calculated by subtracting postintervention ACC scores from preintervention ACC scores. Because of this, more positive ACC improvement scores indicated greater metacognitive improvement.</p> <p>Test grades and predictions between intervention conditions</p> <p> <ephtml> &lt;table&gt;&lt;thead valign="bottom"&gt;&lt;tr&gt;&lt;th&gt;Intervention&lt;/th&gt;&lt;th align="left"&gt;n&lt;/th&gt;&lt;th align="left"&gt;Prediction 1&lt;/th&gt;&lt;th align="left"&gt;Test 1&lt;/th&gt;&lt;th align="left"&gt;Postdiction 1&lt;/th&gt;&lt;th align="left"&gt;Prediction 2&lt;/th&gt;&lt;th align="left"&gt;Test 2&lt;/th&gt;&lt;th align="left"&gt;Postdiction 2&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody valign="top"&gt;&lt;tr&gt;&lt;td&gt;Practice&lt;/td&gt;&lt;td&gt;34&lt;/td&gt;&lt;td&gt;81.91 (7.96)&lt;/td&gt;&lt;td&gt;39.56 (15.34)&lt;/td&gt;&lt;td&gt;60.59 (19.57)&lt;/td&gt;&lt;td&gt;62.64 (17.85)&lt;/td&gt;&lt;td&gt;36.03 (15.51)&lt;/td&gt;&lt;td&gt;53.18 (19.31)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Saliency&lt;/td&gt;&lt;td&gt;33&lt;/td&gt;&lt;td&gt;76.82 (10.81)&lt;/td&gt;&lt;td&gt;42.86 (12.71)&lt;/td&gt;&lt;td&gt;56.42 (15.00)&lt;/td&gt;&lt;td&gt;46.06 (14.56)&lt;/td&gt;&lt;td&gt;40.15 (14.97)&lt;/td&gt;&lt;td&gt;44.46 (18.47)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Motivation&lt;/td&gt;&lt;td&gt;35&lt;/td&gt;&lt;td&gt;79.37 (11.96)&lt;/td&gt;&lt;td&gt;34.43 (15.38)&lt;/td&gt;&lt;td&gt;55.89 (20.91)&lt;/td&gt;&lt;td&gt;57.26 (18.38)&lt;/td&gt;&lt;td&gt;37.14 (13.13)&lt;/td&gt;&lt;td&gt;53.62 (20.29)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Incentive&lt;/td&gt;&lt;td&gt;38&lt;/td&gt;&lt;td&gt;75.79 (11.48)&lt;/td&gt;&lt;td&gt;36.36 (18.34)&lt;/td&gt;&lt;td&gt;55.39 (18.97)&lt;/td&gt;&lt;td&gt;54.19 (17.58)&lt;/td&gt;&lt;td&gt;33.16 (16.20)&lt;/td&gt;&lt;td&gt;50.24 (20.58)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Reflection&lt;/td&gt;&lt;td&gt;39&lt;/td&gt;&lt;td&gt;77.95 (13.21)&lt;/td&gt;&lt;td&gt;44.13 (16.37)&lt;/td&gt;&lt;td&gt;56.026 (17.21)&lt;/td&gt;&lt;td&gt;60.51 (17.39)&lt;/td&gt;&lt;td&gt;36.54 (15.02)&lt;/td&gt;&lt;td&gt;58.08 (16.21)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Control&lt;/td&gt;&lt;td&gt;39&lt;/td&gt;&lt;td&gt;79.92 (8.79)&lt;/td&gt;&lt;td&gt;36.67 (15.01)&lt;/td&gt;&lt;td&gt;60.85 (16.17)&lt;/td&gt;&lt;td&gt;59.49 (16.50)&lt;/td&gt;&lt;td&gt;32.95 (16.25)&lt;/td&gt;&lt;td&gt;60.90 (13.71)&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <p>1 <emph>Note</emph>. Standard deviations are in parentheses.</p> <p>ACC scores and calibration improvement by intervention condition</p> <p> <ephtml> &lt;table&gt;&lt;thead valign="bottom"&gt;&lt;tr&gt;&lt;th&gt;Intervention&lt;/th&gt;&lt;th align="left"&gt;Test 1 ACC (preintervention)&lt;/th&gt;&lt;th align="left"&gt;Test 2 ACC (postintervention)&lt;/th&gt;&lt;th align="left"&gt;ACC improvement&lt;/th&gt;&lt;/tr&gt;&lt;tr&gt;&lt;th align="left"&gt;Prediction 1 ACC&lt;/th&gt;&lt;th align="left"&gt;Postdiction 1 ACC&lt;/th&gt;&lt;th align="left"&gt;Prediction 2 ACC&lt;/th&gt;&lt;th align="left"&gt;Postdiction 2 ACC&lt;/th&gt;&lt;th align="left"&gt;Predictions&lt;/th&gt;&lt;th align="left"&gt;Postdictions&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody valign="top"&gt;&lt;tr&gt;&lt;td&gt;Practice&lt;/td&gt;&lt;td&gt;&amp;#8722;0.90 (18.85)&lt;/td&gt;&lt;td&gt;&amp;#8722;1.45 (16.79)&lt;/td&gt;&lt;td&gt;&amp;#8722;2.30 (18.09)&lt;/td&gt;&lt;td&gt;&amp;#8722;7.45 (12.63)&lt;/td&gt;&lt;td&gt;1.39 (25.52)&lt;/td&gt;&lt;td&gt;6.00 (18.84)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Saliency&lt;/td&gt;&lt;td&gt;&amp;#8722;9.29 (15.12)&lt;/td&gt;&lt;td&gt;&amp;#8722;7.95 (12.71)&lt;/td&gt;&lt;td&gt;&amp;#8722;18.72 (12.54)&lt;/td&gt;&lt;td&gt;&amp;#8722;16.96 (8.46)&lt;/td&gt;&lt;td&gt;9.42 (18.07)&lt;/td&gt;&lt;td&gt;9.01 (14.73)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Motivation&lt;/td&gt;&lt;td&gt;1.69 (17.78)&lt;/td&gt;&lt;td&gt;&amp;#8722;1.69 (17.18)&lt;/td&gt;&lt;td&gt;&amp;#8722;6.84 (13.66)&lt;/td&gt;&lt;td&gt;&amp;#8722;8.53 (&amp;#8722;5.13)&lt;/td&gt;&lt;td&gt;8.53 (19.76)&lt;/td&gt;&lt;td&gt;6.84 (16.80)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Incentive&lt;/td&gt;&lt;td&gt;&amp;#8722;3.84 (18.07)&lt;/td&gt;&lt;td&gt;&amp;#8722;2.03 (16.95)&lt;/td&gt;&lt;td&gt;&amp;#8722;6.40 (15.72)&lt;/td&gt;&lt;td&gt;&amp;#8722;5.13 (18.11)&lt;/td&gt;&lt;td&gt;3.22 (24.248)&lt;/td&gt;&lt;td&gt;3.10 (22.85)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Reflection&lt;/td&gt;&lt;td&gt;&amp;#8722;7.90 (17.76)&lt;/td&gt;&lt;td&gt;&amp;#8722;4.45 (13.91)&lt;/td&gt;&lt;td&gt;&amp;#8722;3.87 (15.35)&lt;/td&gt;&lt;td&gt;&amp;#8722;3.87 (16.97)&lt;/td&gt;&lt;td&gt;&amp;#8722;4.03 (20.18)&lt;/td&gt;&lt;td&gt;&amp;#8722;3.58 (17.88)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Control (uncentered)&lt;/td&gt;&lt;td&gt;43.26 (14.37)&lt;/td&gt;&lt;td&gt;25.72 (16.00)&lt;/td&gt;&lt;td&gt;30.38 (15.91)&lt;/td&gt;&lt;td&gt;28.72 (16.49)&lt;/td&gt;&lt;td&gt;12.87 (17.56)&lt;/td&gt;&lt;td&gt;&amp;#8722;3.00 (17.15)&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <ulist> <item>2 <emph>Note</emph>. Standard deviations are in parentheses. ACC scores represent absolute prediction accuracy compared with control. Higher ACC scores represent more inaccuracy than does control, whereas negative ACCs indicate better accuracy than does control. Intervention efficacy can be measured by ACC improvement between predictions or between postdictions. For every one‐point increase in ACC improvement, predictions became one point more accurate on average, when controlling for natural variation and test fatigue. Control condition data represent (uncentered) absolute calibrations and absolute calibration improvement, around which ACC scores were centered for the intervention conditions.</item> <item>3 Abbreviation: ACC, absolute centered calibration.</item> </ulist> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BU8/01sep19/acp3556-fig-0001.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="acp3556-fig-0001.jpg" title="1 Improvement in absolute controlled calibration for predictions from test 1 to test 2. Note. * indicates significant improvement in absolute controlled calibration (ACC) between prediction 1 and prediction 2, at p &lt; .05. Error bars represent standard mean error. Results are centered around control" /> </p> <p></p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BU8/01sep19/acp3556-fig-0002.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="acp3556-fig-0002.jpg" title="2 Improvement in absolute controlled calibration for postdictions from test 1 to test 2. Note. * indicates significant improvement in absolute controlled calibration (ACC) between postdiction 1 and postdiction 2, at p &lt; .05. Error bars represent standard mean error. Results are centered around control" /> </p> <p></p> <hd id="AN0138393115-21">Change in prediction calibration</hd> <p>We used a 5 × 2 mixed ANOVA to examine the repeated‐measures effect of Intervention Condition and between‐subjects effect of Prediction Time on ACC scores. We compared the effects of the five interventions and the control condition on calibration between prediction 1 and prediction 2 using a 5 × 2 repeated‐measures mixed ANOVA on ACC scores. There was a near‐significant interaction between Prediction Time and Intervention Condition, <emph>F</emph>(<reflink idref="bib5" id="ref67">5</reflink>, 211) = 2.15, <emph>p</emph> = .060, partial <emph>η</emph><sups>2</sups> = .05, on prediction ACC scores. This result suggests that the change in ACC scores varied depending on the type of intervention.</p> <p>Because the effect of Intervention Condition on Prediction Time interaction approached significance, and because there was a significant difference in absolute calibration in the control condition between predictions 1 and 2, <emph>F</emph>(<reflink idref="bib1" id="ref68">1</reflink>, 38) = 20.96, <emph>p</emph> &lt; .001, partial <emph>η</emph><sups>2</sups> = .36, we compared the change in ACC from prediction 1 to prediction 2 for each intervention (Figure ). The Saliency, <emph>F</emph>(<reflink idref="bib1" id="ref69">1</reflink>, 32) = 8.97, <emph>p</emph> = .005, partial <emph>η</emph><sups>2</sups> = .22, and Motivation Warning Lecture, <emph>F</emph>(<reflink idref="bib1" id="ref70">1</reflink>, 34) = 6.52, <emph>p</emph> = .015, partial <emph>η</emph><sups>2</sups> = .16, interventions led to significant calibration improvements from test 1 to test 2. However, the Review, <emph>F</emph>(<reflink idref="bib1" id="ref71">1</reflink>, 33) = 0.10, <emph>p</emph> = .752, partial <emph>η</emph><sups>2</sups> &lt; .01, Incentive, <emph>F</emph>(<reflink idref="bib1" id="ref72">1</reflink>, 36) = 0.64, <emph>p</emph> = .428, partial <emph>η</emph><sups>2</sups> = .02, and Reflection, <emph>F</emph>(<reflink idref="bib1" id="ref73">1</reflink>, 32) = 1.56, <emph>p</emph> = .220, partial <emph>η</emph><sups>2</sups> = .04, interventions did not (see Table ). Thus, only the Saliency and Motivation Warning Lecture interventions improved calibration relative to control.</p> <hd id="AN0138393115-22">Change in postdiction calibration</hd> <p>We analyzed postdiction data in the same way as prediction data, using a 5 × 2 mixed ANOVA comparing ACC scores by Intervention Condition and Postdiction Time. There was a significant interaction between Intervention Condition and Postdiction Time, <emph>F</emph>(<reflink idref="bib5" id="ref74">5</reflink>, 212) = 2.41, <emph>p</emph> = .037, partial <emph>η</emph><sups>2</sups> = .05, indicating that ACC scores between postdictions 1 and 2 varied depending on the intervention participants received. Given that absolute calibration postdiction scores did not differ in the control condition between test 1 and test 2, <emph>t</emph>(<reflink idref="bib38" id="ref75">38</reflink>) = 1.19, <emph>p</emph> = .282, <emph>d</emph> = .03, these results indicate that at least one of the intervention conditions produced change in postdiction accuracy.</p> <p>We examined improvement in postdiction accuracy from test 1 to test 2 between the five intervention conditions using five different repeated‐measures ANOVAs (summarized in Figure ). Mirroring the pattern observed for prediction calibration, participants in the Salient feedback intervention, <emph>F</emph>(<reflink idref="bib1" id="ref76">1</reflink>, 32) = 12.35, <emph>p</emph> = .001, partial <emph>η</emph><sups>2</sups> = .28, and in the Motivation Warning Lecture intervention, <emph>F</emph>(<reflink idref="bib1" id="ref77">1</reflink>, 34) = 5.80, <emph>p</emph> = .022, partial <emph>η</emph><sups>2</sups> = .15, improved their postdiction ACC scores from test 1 to test 2 (see Figure ). However, those in the Review, <emph>F</emph>(<reflink idref="bib1" id="ref78">1</reflink>, 33) = 3.45, <emph>p</emph> = .072, partial <emph>η</emph><sups>2</sups> = .10, Incentive, <emph>F</emph>(<reflink idref="bib1" id="ref79">1</reflink>, 37) = 0.70, <emph>p</emph> = .408, partial <emph>η</emph><sups>2</sups> = .02, and Reflection interventions, <emph>F</emph>(<reflink idref="bib1" id="ref80">1</reflink>, 38) = 1.56, <emph>p</emph> = .219, partial <emph>η</emph><sups>2</sups> = .04, did not.</p> <hd id="AN0138393115-23">DISCUSSION</hd> <p>To improve metacognition, investigators have designed a wide variety of interventions with inconsistent success (Foster et al., [<reflink idref="bib10" id="ref81">10</reflink>]; Hacker et al., [<reflink idref="bib15" id="ref82">15</reflink>]; Nietfeld et al., [<reflink idref="bib28" id="ref83">28</reflink>], [<reflink idref="bib29" id="ref84">29</reflink>]). We compared five such interventions to identify which paradigms result in metacognitive improvement. We found that only two interventions, salient feedback and a motivation warning lecture, produced significant improvements in test grade prediction accuracy.</p> <hd id="AN0138393115-24">Successful interventions</hd> <p>The saliency condition produced the greatest amount of calibration improvement compared with baseline. As this condition was based on the most common intervention, feedback, the results are consistent with a number of findings showing that feedback can improve calibration accuracy (Flannelly, [<reflink idref="bib9" id="ref85">9</reflink>]; Hacker et al., [<reflink idref="bib14" id="ref86">14</reflink>]; Miller &amp; Geraci, [<reflink idref="bib24" id="ref87">24</reflink>]; Nietfeld et al., [<reflink idref="bib29" id="ref88">29</reflink>]). However, not all feedback‐based interventions produce improvement (Bol et al., [<reflink idref="bib2" id="ref89">2</reflink>]; Foster et al., [<reflink idref="bib10" id="ref90">10</reflink>]; Nietfeld et al., [<reflink idref="bib28" id="ref91">28</reflink>]). Further investigations must focus on what characteristics of feedback interventions that are most effective for improving calibration. Although some have suggested that sufficiently "strong" feedback procedures are needed to produce improvements in calibration (Nietfeld et al., [<reflink idref="bib29" id="ref92">29</reflink>]; Hacker, Bol, &amp; Keener, [<reflink idref="bib16" id="ref93">16</reflink>]), what is needed is information about what constitutes a strong or effective feedback procedure. Repeated exposure to feedback does not seem to be effective (Bol et al., [<reflink idref="bib2" id="ref94">2</reflink>]; Foster et al., [<reflink idref="bib10" id="ref95">10</reflink>]). Other studies have attempted more extensive training plans involving feedback alongside other interventions (Flannelly, [<reflink idref="bib9" id="ref96">9</reflink>]; Hacker et al., [<reflink idref="bib14" id="ref97">14</reflink>]; Miller &amp; Geraci, [<reflink idref="bib24" id="ref98">24</reflink>]; Nietfeld et al., [<reflink idref="bib29" id="ref99">29</reflink>]), but because these studies included other variables in addition to feedback, they cannot be used to identify what type of feedback can produce calibration improvement. The current study showed that salient feedback provided immediately before a subsequent prediction improved calibration. We hypothesize that the "strength" of this intervention, beyond other types of feedback, was due to its immediacy in relation to further tests and predictions. Many studies provide students with feedback shortly after an initial exam, but few make that feedback salient immediately before a subsequent exam, or during future performance predictions. Instead, this intervention involved feedback, printed in large font, provided to each participant as they made predictions and postdictions about their performance on the second exam. Further, participants had listened to the study proctor explain the feedback form and encourage them to think about the potential benefits of better prediction accuracy, reducing the chances that they could simply ignore or disregard the feedback. Though it is difficult to speculate on which of these "saliency" features are necessary or sufficient to elicit calibration improvement, we propose that feedback alone, as long as it is sufficiently explicit and occurs immediately prior to a subsequent prediction, may be an effective method for improving calibration.</p> <p>We also found that a motivation warning lecture about the potentially biasing effects of desired performance significantly reduced metacognitive error. This intervention encouraged participants to rely on academic information instead of grade desires and produced significant improvements in calibration without the need for direct feedback. This finding replicated previous research (Saenz et al., [<reflink idref="bib32" id="ref100">32</reflink>]), showing that a similar warning was effective for improving calibration in the classroom. What aspect of this lecture‐based intervention elicited calibration change? We propose that the instruction to avoid motivational information and focus on academic information when making grade predictions was the defining factor in improving calibration. Could participants have improved their calibration as a demand characteristic of the motivation warning lecture? Although the present intervention included information about the overconfident nature of predictions, it seems improbable that the motivation intervention amounted to simply asking participants to lower their performance predictions. Simply informing students about the overconfident nature of predictions has failed to improve calibration in previous research (Miller &amp; Geraci, [<reflink idref="bib24" id="ref101">24</reflink>]). In contrast, Saenz et al. ([<reflink idref="bib32" id="ref102">32</reflink>]) found that following a similar lecture that did include a warning about motivational biases, students' overconfidence decreased, and predictions shifted to relying on educational factors (such as prior test performance and amount of study) rather than motivations, suggesting that the warning did not lead students to arbitrarily lower their predictions. The current results show that a warning without individualized feedback is sufficient to improve calibration.</p> <p>Although the motivation warning lecture intervention was successful in improving calibration scores, it is notable that the effect size was only slightly lower than that of the saliency intervention. One might have predicted that the effect of the motivation warning lecture would be smaller than the effect of the salient feedback because motivations should play a lesser role in the laboratory than in other settings, such as a classroom (e.g., Gramzow, Johnson, &amp; Willard, [<reflink idref="bib12" id="ref103">12</reflink>]; Gramzow, Elliot, Asher, &amp; McGregor, [<reflink idref="bib11" id="ref104">11</reflink>]; Hacker et al., [<reflink idref="bib14" id="ref105">14</reflink>]; Saenz et al., [<reflink idref="bib32" id="ref106">32</reflink>]). Yet the motivation warning lecture intervention did improve calibration, suggesting that people felt at least some motivation to evaluate themselves positively, even in a laboratory setting. Another reason one might have expected the effect of the motivation to be small in the motivation warning lecture intervention condition is that the video lecture encouraged participants to attempt to improve their prediction accuracy with the goal of better test preparation. Again, because the intervention was used in a laboratory setting, participants did not gain anything from improving their accuracy and could not be expected to improve their test performance given that there was no time for additional study. Despite these aspects of the design, the motivation warning lecture intervention was successful at improving prediction accuracy on a level approaching the saliency intervention, which may speak to its robustness. Motivation may be only one of multiple factors that combine to impair students' metacognitive accuracy. For example, if people harbor inaccurate ideas about others' performance (as evidenced by Tirso &amp; Geraci, [<reflink idref="bib38" id="ref107">38</reflink>]), they may misunderstand their relative standing and make erroneous self‐monitoring judgments. Further investigations may seek to identify other factors plaguing knowledge‐monitoring judgments.</p> <p>The current study used interventions presented immediately before and after tests and predictions, but other intervention studies often include a delay of days or weeks between tests given the natural delay between classroom exams (Foster et al., [<reflink idref="bib10" id="ref108">10</reflink>]; Nietfeld et al., [<reflink idref="bib28" id="ref109">28</reflink>]). This delay may make a substantial difference in calibration that may go beyond the effects associated with intervention conditions. For example, performance predictions may be the result of two forms of metacognitive feedback: feedback from test scores or interventions and feedback produced as a by‐product of study or test preparation. The present study examined only the former—feedback from scores and interventions—as study was not possible. By eliminating the delay between interventions and tests, we may observe the effect of the interventions, without any internal feedback or behavioral effects. It may be that the saliency and motivation warning lecture interventions would be similarly efficacious under a delayed paradigm. The motivation warning lecture paradigm has been employed in a delayed, classroom study (Saenz et al., [<reflink idref="bib32" id="ref110">32</reflink>]), whereas the saliency intervention was employed, in a form, by Miller and Geraci ([<reflink idref="bib24" id="ref111">24</reflink>]), who found significant calibration improvement after presenting test grades alongside predictions (although this study also offered incentives for improved prediction accuracy). However, we speculate that the efficacy of the saliency intervention was based on its proximity to the next exam and predictions and may be diminished with a longer delay between feedback and later tests because the delay would reduce the saliency of the feedback. Further investigation is needed to determine the relationship between test delay and interventions.</p> <hd id="AN0138393115-25">Unsuccessful interventions</hd> <p>It may be useful to consider aspects of the failed interventions: the review intervention, the incentive intervention, and the reflection intervention. Although the successful interventions suggest that feedback, whether direct or indirect, improves calibration, we did not find a significant effect of the review intervention, which included a form of feedback. However, it would be unreasonable to say that test practice does not benefit calibration. Most practice tests cover learned material much more extensively than the four‐question, 10‐min, intervention used in the current study. It is also possible that the review intervention did not improve calibration because participants already learned all they could from testing practice before receiving the intervention. That is, the first logical reasoning test may have served as a full‐length practice test for the second test, providing students with information on what to expect on the second test, and how they might perform. The data support this interpretation; after taking test 1, subsequent grade predictions dropped considerably in all conditions before any interventions were administered (see Table ). This drop represented the single largest improvement in calibration observed in the current study. Thus, completing four additional practice questions during the review intervention may have yielded no further benefit to calibration because these tests offered participants no further insight regarding the tests or their abilities.</p> <p>Both the incentive and reflection interventions were ineffective at producing calibration change. Participants in the incentive condition were offered a chance for a $50 reward for accurate predictions, yet they did not improve their calibration. It may be that incentives are not what people need to make better predictions, as they do not teach people what they know or do not know, as in the salient feedback condition, or remove barriers to accurate predictions as in the motivation warning lecture intervention. Further, poor calibration may not be caused by a lack of effort. Of course, it is possible that in the incentive intervention, participants simply did not find a 1/40 chance to win $50 as enough of an incentive to motivate them to change their predictions. The current data suggest that an incentive alone does not produce beneficial calibration change. It may be that incentives alone are ineffective but can serve as catalysts when paired with other interventions (e.g., feedback and warnings), motivating people to take the intervention to heart.</p> <p>Much like the incentive intervention, the reflection intervention did not produce a change in calibration, possibly because participants were not given a method for improving prediction accuracy. Instead, participants were simply encouraged to think carefully about their predictions. Like many interventions that employ repeated feedback (Foster et al., [<reflink idref="bib10" id="ref112">10</reflink>], Hacker, Bol, &amp; Keener, [<reflink idref="bib16" id="ref113">16</reflink>], Nietfeld et al., [<reflink idref="bib28" id="ref114">28</reflink>]), the reflection condition failed to improve calibration, further suggesting that calibration does not improve through either repeated prediction attempts or extended prediction contemplation. Instead, participants gradually increased predictions throughout the reflection intervention predictions (Figure ), peaking at a significantly higher point (<emph>M</emph> = 62.72, <emph>SD</emph> = 16.08), <emph>t</emph>(<reflink idref="bib38" id="ref115">38</reflink>) = 2.96, <emph>p</emph> = .005, Cohen's <emph>d</emph> = .47, than their respective postdiction 2 (<emph>M</emph> = 56.03, <emph>SD</emph> = 17.21). Further investigation is needed to understand why predictions increased during the reflection intervention.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BU8/01sep19/acp3556-fig-0003.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="acp3556-fig-0003.jpg" title="3 Predictions across reflection intervention. Note. Prediction 1 omitted to show detail. Y axis represents estimates of test performance out of 100% correct on associated tests" /> </p> <p></p> <p>Might the unsuccessful intervention paradigms elicit more promising results under a delayed‐testing design? We hypothesize that the review condition may be more effective under a delayed paradigm, where prior test performance may be less salient and test practice may serve as a form of feedback or reminder of prior test performance. It is difficult to say whether the incentive condition would have led to different results under delayed‐testing conditions, as one may consider such a design inherent in a classroom setting where tests are naturally delayed and students have incentives to make accurate metacognitive judgments to sufficiently prepare for tests. However, the reflection intervention may elicit different results under a delayed paradigm. Did participants' calibration deteriorate under the reflection condition because of the number of predictions made, or did the amount of time spent thinking about their predictions make participants more confident? Under delayed‐testing conditions, such as those experienced by students in most university classes, extended rumination on future test scores may affect performance prediction accuracy, potentially due to the same motivations that drive students to make overconfident judgments (Saenz et al., [<reflink idref="bib32" id="ref116">32</reflink>]). Further study is needed to observe the effects of the present interventions under testing conditions with greater delays.</p> <hd id="AN0138393115-27">Limitations</hd> <p>The conclusions one can draw from the current study may be limited by the fact that the study was conducted in a controlled laboratory setting. This setting is helpful for isolating the features of interventions that affect calibration, but aspects of the laboratory study may not translate to other settings. For example, the salient feedback intervention relied on providing people with feedback that is <emph>salient</emph> at the time of a prediction. Doing so may be difficult in a classroom setting, as some students presumably make these types of predictions automatically throughout the days preceding a test and when choosing when and how much to study.</p> <p>We note that none of our intervention conditions elicited performance change (Table ). They were not meant to, as participants did not have an opportunity to study following the intervention. However, there is evidence that the motivation warning lecture intervention can elicit both calibration change and performance improvement in a classroom setting (see Saenz et al., [<reflink idref="bib32" id="ref117">32</reflink>]). One condition that did offer a modest opportunity for learning was the review intervention condition, though there was no improvement in performance. A more extensive review intervention may be necessary to improve performance and/or calibration.</p> <p>As mentioned, the greatest improvement in calibration was not produced by an intervention but simply by taking the exam for the first time. After taking test 1, participants' calibration improved by about 20 points, a change that remained stable across further judgments (Table ). Postdictions are more accurate than predictions because people learn what they know and do not know after taking an exam (Maki, [<reflink idref="bib22" id="ref118">22</reflink>]; McCormic, [<reflink idref="bib23" id="ref119">23</reflink>]; Pierce &amp; Smith, [<reflink idref="bib30" id="ref120">30</reflink>]; Pressley &amp; Ghatala, [<reflink idref="bib31" id="ref121">31</reflink>]). Interestingly, although calibration improved from prediction one to postdiction one, no calibration improvement occurred between any subsequent prediction or postdiction in the control condition, potentially because every judgment after the initial exam was a postdiction in the sense that participants had already experienced related test material. Such results suggest that participants obtained metacognitive information from taking the first test, but no further information from the second test. Given the sizable calibration improvement in the control condition, our intervention effects may be conservative estimates of the potential effects. These interventions may be more effective if students are given time to study. The test review intervention may have been particularly limited by our procedure, as participants may have already gained all the test practice benefits available before ever receiving the review intervention.</p> <p>The current study was designed to examine interventions to improve calibration, in the form of overconfidence. We were interested in overconfidence because overconfidence has been linked with lower test performance (e.g., Bol et al., [<reflink idref="bib2" id="ref122">2</reflink>]; Kruger &amp; Dunning, [<reflink idref="bib21" id="ref123">21</reflink>]; Nietfeld et al., [<reflink idref="bib28" id="ref124">28</reflink>], [<reflink idref="bib29" id="ref125">29</reflink>]). As such, the current study was designed to elicit a high level of overconfidence, such that participants would have ample opportunity to improve their calibration. However, people can also show poor calibration in the form of underconfidence. Underconfidence may be a desirable form of calibration error in the classroom because it may elicit greater study behavior, and because underconfidence has been linked with higher test performance (Kruger &amp; Dunning, [<reflink idref="bib21" id="ref126">21</reflink>]). One might wonder if the same interventions that reduce overconfidence would reduce underconfidence. The current study cannot speak to this question, as only three of the 218 participants exhibited underconfidence, but future studies might examine potential methods for improving calibration in conditions of underconfidence.</p> <hd id="AN0138393115-28">Conclusions</hd> <p>Educators may choose to employ the successful interventions reported here in the classroom. To attempt to implement an intervention akin to the saliency intervention used in this study, students could be asked to predict their grades immediately before tests, and instructors could post those predictions alongside grades to make them salient, following the methodology used by Miller and Geraci ([<reflink idref="bib24" id="ref127">24</reflink>]). Although Miller and Geraci originally included incentives, the present results suggest that salient presentation of inaccurate predictions alongside grades may be sufficient to produce an improvement in calibration. Instructors might also let students know about the pitfalls of overconfidence that may be associated with motivations. Such conversations fit neatly in the curricula of many psychology and education‐oriented classes and would not be out of place in any university course. Whereas high confidence may be motivating in many aspects of life (Gramzow et al., [<reflink idref="bib11" id="ref128">11</reflink>]; Gramzow et al., [<reflink idref="bib12" id="ref129">12</reflink>]; Taylor &amp; Brown, [<reflink idref="bib35" id="ref130">35</reflink>]), there are benefits to accurately assessing one's knowledge and abilities. However, it should be noted that although the current study has demonstrated the potential effectiveness of these interventions, these interventions remain untested in the classroom. Additional research in classroom settings is required to determine how well these interventions translate to classroom settings before widespread adoption could be encouraged.</p> <p>To conclude, the current study was designed to directly compare the efficacy of various metacognitive intervention strategies that have been individually tested in past studies across differing conditions. Of the five interventions examined, only the saliency and motivation warning lecture interventions significantly improved calibration relative to control. These findings suggest that providing salient feedback regarding past performance and prediction accuracy shortly before people make subsequent metacognitive judgments may be effective ways of improving students' metacognitive accuracy. Additionally, shifting people's focus away from motivational factors, such as the level of performance they wish to attain, may be another effective way of improving metacognitive accuracy. Because the motivation warning lecture condition did not involve any form of feedback, we can conclude that feedback is sufficient, but not necessary, to elicit calibration improvement. However, the current results may provide a conservative estimate of the effectiveness of these various intervention conditions because the intervention conditions were compared with a very stringent control condition, one in which participants had already improved their metacognitive judgments by about 20 points (see Table ). This means that the current study tested the effectiveness of interventions over and above a sizable metacognitive improvement that may not reflect classroom conditions. Further investigations may also seek to determine which aspects of interventions are necessary for calibration change, or whether there are other types of interventions that may elicit calibration change without motivation warnings, or feedback. In addition, future research might more closely examine the role of feedback in conjunction with these interventions. In the current study, participants did not receive performance feedback in any of the interventions, with the exception of the saliency condition. It may be that feedback in conjunction with one or more intervention conditions may elicit metacognitive change that was not observed in this study. For example, although participants in the incentive condition did not improve metacognitive accuracy, existing data suggest that incentives may improve calibration when combined with performance feedback (Miller &amp; Geraci, [<reflink idref="bib24" id="ref131">24</reflink>]). Further investigations aside, the two successful interventions showed that there are indeed short, classroom‐appropriate interventions that can be used to improve metacognitive accuracy.</p> <hd id="AN0138393115-29">CONFLICT OF INTEREST</hd> <p>The authors certify that they have no affiliation with or involvement in any organization or entity with any financial interest in the subject matter or materials discussed in this manuscript.</p> <p>A APPENDIX LOGICAL REASONING EXAM SAMPLE QUESTIONS</p> <p>Questions 1–2</p> <p>A company employee generates a series of five‐digit product codes in accordance with the following rules:</p> <p>The codes use the digits 0, 1, 2, 3, and 4, and no others. Each digit occurs exactly once in any code. The second digit has a value exactly twice that of the first digit.</p> <p>The value of the third digit is less than the value of the fifth digit.</p> <p></p> <ulist> <item> If the last digit of an acceptable product code is 1, it must be true that the (A) first digit is 2 (B) second digit is 0 (C) third digit is 3 (D) fourth digit is 4 (E) fourth digit is 0</item> <p></p> <item> Which one of the following must be true about any acceptable product code? (A) The digit 1 appears in some position before the digit 2. (B) The digit 1 appears in some position before the digit 3. (C) The digit 2 appears in some position before the digit 3. (D) The digit 3 appears in some position before the digit 0. (E) The digit 4 appears in some position before the digit 3.</item> </ulist> <p>Question 3</p> <p></p> <ulist> <item> Situation: Someone living in a cold climate buys a winter coat that is stylish but not warm in order to appear sophisticated. Analysis: People are sometimes willing to sacrifice sensual comfort or pleasure for the sake of appearances. The analysis provided for the situation above is most appropriate for which one of the following situations? (A) A person buys an automobile to commute to work even though public transportation is quick and reliable. (B) A parent buys a car seat for a young child because it is more colorful and more comfortable for the child than the other car seats on the market, though no safer. (C) A couple buys a particular wine even though their favorite wine is less expensive and better tasting because they think it will impress their dinner guests. (D) A person sets her thermostat at a low temperature during the winter because she is concerned about the environmental damage caused by using fossil fuels to heat her home. (E) An acrobat convinces the circus that employs him to purchase an expensive outfit for him so that he can wear it during his act to impress the audience.</item> </ulist> <p>B APPENDIX SAMPLE INTERVENTION MATERIALS</p> <hd1 id="AN0138393115-30">Practice Intervention</hd1> <hd1 id="AN0138393115-31">Sample Question</hd1> <p>1. The past three consecutive women's U.S. tennis champions have all changed to Wilson's new line of tennis rackets, exclusively made of oak wood for greater strength and durability. If this is the case, do not you think it's time to improve your tennis swing and trade your old racket in for a Wilson? Which of the following claims is not made and cannot be used in conclusion to the above advertisement?</p> <p>A. Previous U.S. tennis champions know a considerable amount about their equipment and the sport of tennis.</p> <p>B. Rackets that are strengthened by oak wood are used exclusively in Wilson's new rackets.</p> <p>C. Oak‐wood‐strengthened rackets help to make tennis rackets durable and stronger, allowing the player to make powerful swings.</p> <p>D. With Wilson's rackets, you will improve your tennis playing.</p> <p>E. The status achieved by the past three consecutive women's U.S. tennis championships was due to the assistance of Wilson's rackets.</p> <p>Do you think you will get this question right? (circle one)</p> <p>Yes</p> <p>No</p> <p>How confident are you that you will get this question right? (0%–100%)</p> <p>___________</p> <p>‐‐‐‐‐‐‐‐‐‐‐‐</p> <hd1 id="AN0138393115-32">Feedback and Survey</hd1> <p> <bold>1. E:</bold> Champions "have all changed to" Wilson's new rackets; they did not win past championships with them. Champions are knowledgeable about tennis and their equipment (A): The ad uses their choosing Wilson's as an example to follow. It indicates exclusive use of oak in Wilson's new line (B). Durability and strength are named as oak's benefits; from the suggestion these will "improve your tennis swing," we can infer "powerful swings" (C) and "you will improve your tennis playing" (D).</p> <p>I got this question (circle one):</p> <p></p> <p>• Correct</p> <p></p> <ulist> <item> Incorrect</item> </ulist> <p>My prediction about my accuracy was (circle one):</p> <p></p> <p>• Correct</p> <p></p> <ulist> <item> Incorrect</item> </ulist> <p>Based on these results, I might want to ________ my confidence (circle one):</p> <p></p> <ulist> <item> Increase</item> <p></p> <item> Decrease</item> </ulist> <hd1 id="AN0138393115-33">Saliency Intervention</hd1> <p>Your Grade Prediction 70</p> <p>Your Actual Grade 60</p> <p>Your Calibration Score 10</p> <p></p> <p> <ephtml> &lt;table&gt;&lt;tbody valign="top"&gt;&lt;tr&gt;&lt;td&gt;&lt;bold&gt;Incentive Intervention&lt;/bold&gt;&lt;/td&gt;&lt;td&gt;&lt;bold&gt;Reflection Intervention&lt;/bold&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td /&gt;&lt;td /&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; </ephtml> </p> <p>4 <emph>Note</emph>. Slides and a transcript of the <bold>Motivational Warning Lecture</bold> are available at https://osf.io/shjad/?view_only=0faf6b5f43a848a596b5e35cacad3ac9.</p> <ref id="AN0138393115-34"> <title> REFERENCES </title> <blist> <bibl id="bib1" idref="ref53" type="bt">1</bibl> <bibtext> Bol, L., &amp; Hacker, D. J. (2001). A comparison of the effects of practice tests and traditional review on performance and calibration. The Journal of Experimental Education, 69, 133 – 151. https://doi.org/10.1080/00220970109600653</bibtext> </blist> <blist> <bibl id="bib2" idref="ref1" type="bt">2</bibl> <bibtext> Bol, L., Hacker, D. J., O'Shea, P., &amp; Allen, D. (2005). The influence of overt practice, achievement level, and explanatory style on calibration accuracy and performance. The Journal of Experimental Education, 73, 269 – 290. https://doi.org/10.3200/JEXE.73.4.269‐290</bibtext> </blist> <blist> <bibl id="bib3" idref="ref44" type="bt">3</bibl> <bibtext> Callender, A. A., Franco‐Watkins, A. M., &amp; Roberts, A. S. (2016). Improving metacognition in the classroom through instruction, training, and feedback. Metacognition and Learning, 11, 215 – 235. https://doi.org/10.1007/s11409‐015‐9142‐6</bibtext> </blist> <blist> <bibl id="bib4" idref="ref19" type="bt">4</bibl> <bibtext> Dunlosky, J., &amp; Rawson, K. A. (2012). Overconfidence produces underachievement: Inaccurate self evaluations undermine students' learning and retention. Learning and Instruction, 22, 271 – 280. https://doi.org/10.1016/j.learninstruc.2011.08.003</bibtext> </blist> <blist> <bibl id="bib5" idref="ref50" type="bt">5</bibl> <bibtext> Dunlosky, J., Rawson, K. A., &amp; McDonald, S. L. (2002). Influence of practice tests on the accuracy of predicting memory performance for paired associates, sentences, and text material. In T. J. Perfect, &amp; B. L. Schwartz (Eds.), Applied metacognition (pp. 68 – 92). https://doi.org/10.1017/CBO9780511489976.005</bibtext> </blist> <blist> <bibl id="bib6" idref="ref2" type="bt">6</bibl> <bibtext> Dunning, D., Johnson, K., Ehrlinger, J., &amp; Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12, 83 – 87. https://doi.org/10.1111/1467‐8721.01235</bibtext> </blist> <blist> <bibl id="bib7" idref="ref3" type="bt">7</bibl> <bibtext> Ehrlinger, J., Johnson, K., Banner, M., Dunning, D., &amp; Kruger, J. (2008). Why the unskilled are unaware: Further explorations of (absent) self‐insight among the incompetent. Organizational Behavior and Human Decision Processes, 105, 98 – 121. https://doi.org/10.1016/j.obhdp.2007.05.002</bibtext> </blist> <blist> <bibl id="bib8" idref="ref25" type="bt">8</bibl> <bibtext> Everson, H. T., &amp; Tobias, S. (1998). The ability to estimate knowledge and performance in college: A metacognitive analysis. Instructional Science, 26, 65 – 79. https://doi.org/10.1023/A:1003040130125</bibtext> </blist> <blist> <bibl id="bib9" idref="ref85" type="bt">9</bibl> <bibtext> Flannelly, L. T. (2001). Using feedback to reduce students' judgment bias on test questions. Journal of Nursing Education, 40, 10 – 16. https://doi.org/10.3928/0148‐4834‐20010101‐05</bibtext> </blist> <blist> <bibtext> Foster, N. L., Was, C. A., Dunlosky, J., &amp; Isaacson, R. M. (2016). Even after thirteen class exams, students are still overconfident: The role of memory for past exam performance in student predictions. Metacognition and Learning, 12, 1 – 19. https://doi.org/10.1007/s11409‐016‐9158‐6</bibtext> </blist> <blist> <bibtext> Gramzow, R. H., Elliot, A. J., Asher, E., &amp; McGregor, H. A. (2003). Self‐evaluation bias and academic performance: Some ways and some reasons why. Journal of Research in Personality, 37, 41 – 61. https://doi.org/10.1016/S0092‐6566(02)00535‐4</bibtext> </blist> <blist> <bibtext> Gramzow, R. H., Johnson, C. S., &amp; Willard, G. (2014). Boasts are a boost: Achievement prime self‐reactivity predicts subsequent academic performance. Journal of Personality and Social Psychology, 106, 458 – 468. https://doi.org/10.1037/a0035560</bibtext> </blist> <blist> <bibtext> Gutierrez, A. P., &amp; Schraw, G. (2015). Effects of strategy training and incentives on students' performance, confidence, and calibration. The Journal of Experimental Education, 83, 386 – 404. https://doi.org/10.1080/00220973.2014.907230</bibtext> </blist> <blist> <bibtext> Hacker, D. J., Bol, L., &amp; Bahbahani, K. (2008). Explaining calibration accuracy in classroom contexts: The effects of incentives, reflection, and explanatory style. Metacognition and Learning, 3, 101 – 121. https://doi.org/10.1007/s11409‐008‐9021‐5</bibtext> </blist> <blist> <bibtext> Hacker, D. J., Bol, L., Horgan, D. D., &amp; Rakow, E. A. (2000). Test prediction and performance in a classroom context. Journal of Educational Psychology, 92, 160 – 170. https://doi.org/10.1037/0022‐0663.92.1.160</bibtext> </blist> <blist> <bibtext> Hacker, D. J., Bol, L., &amp; Keener, M. C. (2008). Metacognition in education: A focus on calibration. In J. Dunlosky, &amp; R. A. Bjork (Eds.), Handbook of metamemory and memory (pp. 429 – 455). New York, NY : Taylor &amp; Francis Group. https://doi.org/10.4324/9780203805503.ch22</bibtext> </blist> <blist> <bibtext> Hartwig, M. K., &amp; Dunlosky, J. (2014). The contribution of judgment scale to the unskilled‐and‐unaware phenomenon: How evaluating others can exaggerate over‐ (and under‐) confidence. Memory &amp; Cognition, 42, 164 – 173. https://doi.org/10.3758/s13421‐013‐0351‐4</bibtext> </blist> <blist> <bibtext> Helzer, E. G., &amp; Dunning, D. (2012). Why and when peer prediction is superior to self‐prediction: The weight given to future aspiration versus past achievement. Journal of Personality and Social Psychology, 103, 38 – 53. https://doi.org/10.1037/a0028124</bibtext> </blist> <blist> <bibtext> Hogarth, R. M., Gibbs, B. J., McKenzie, C. R., &amp; Marquis, M. A. (1991). Learning from feedback: Exactingness and incentives. Journal of Experimental Psychology: Learning, Memory, and Cognition, 17, 734.</bibtext> </blist> <blist> <bibtext> Kelemen, W. L., Winningham, R. G., &amp; Weaver, C. A. III (2007). Repeated testing sessions and scholastic aptitude in college students' metacognitive accuracy. European Journal of Cognitive Psychology, 19, 689 – 717. https://doi.org/10.1080/09541440701326170</bibtext> </blist> <blist> <bibtext> Kruger, J., &amp; Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self‐assessments. Journal of Personality and Social Psychology, 77, 1121 – 1134. https://doi.org/10.1037/0022‐3514.77.6.1121</bibtext> </blist> <blist> <bibtext> Maki, R. H. (1998). Test predictions over text material. In D. J. Hacker, J. Dunlosky, &amp; A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 117 – 144). Mahwah, NJ : Erlbaum.</bibtext> </blist> <blist> <bibtext> McCormic, C. B. (2003). Metacognition and learning. In W. M. Reynolds, &amp; G. E. Miller (Eds.), Handbook of psychology: Vol. 7, educational psychology (pp. 79 – 102). New York : Wiley.</bibtext> </blist> <blist> <bibtext> Miller, T. M., &amp; Geraci, L. (2011a). Training metacognition in the classroom: The influence of incentives and feedback on exam predictions. Metacognition and Learning, 6, 303 – 314. https://doi.org/10.1007/s11409‐011‐9083‐7</bibtext> </blist> <blist> <bibtext> Miller, T. M., &amp; Geraci, L. (2011b). Unskilled but aware: Reinterpreting overconfidence in low‐performing students. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37, 502 – 506. https://doi.org/10.1037/a0021802</bibtext> </blist> <blist> <bibtext> Miller, T. M., &amp; Geraci, L. (2014). Improving metacognitive accuracy: How failing to retrieve practice items reduces overconfidence. Consciousness &amp; Cognition, 29, 131 – 140. https://doi.org/10.1016/j.concog.2014.08.008</bibtext> </blist> <blist> <bibtext> Miller, T. M., &amp; Geraci, L. (2016). The influence of retrieval practice on metacognition: The contribution of analytic and non‐analytic processes. Consciousness &amp; Cognition, 42, 41 – 50. https://doi.org/10.1016/j.concog.2016.03.010</bibtext> </blist> <blist> <bibtext> Nietfeld, J. L., Cao, L., &amp; Osborne, J. W. (2005). Metacognitive monitoring accuracy and student performance in the postsecondary classroom. Journal of Experimental Education, 74, 7 – 28.</bibtext> </blist> <blist> <bibtext> Nietfeld, J. L., Cao, L., &amp; Osborne, J. W. (2006). The effect of distributed monitoring exercises and feedback on performance, monitoring accuracy, and self‐efficacy. Metacognition and Learning, 1, 159 – 179. https://doi.org/10.1007/s10409‐006‐9595‐6</bibtext> </blist> <blist> <bibtext> Pierce, B. H., &amp; Smith, S. M. (2001). The postdiction superiority effect in metacomprehension of text. Memory &amp; Cognition, 29, 62 – 67. https://doi.org/10.3758/BF03195741</bibtext> </blist> <blist> <bibtext> Pressley, M., &amp; Ghatala, E. S. (1990). Self‐regulated learning: Monitoring learning from text. Educational Psychologist, 25, 19 – 33. https://doi.org/10.1207/s15326985ep2501_3</bibtext> </blist> <blist> <bibtext> Saenz, G. D., Geraci, L., Miller, T. M., &amp; Tirso, R. (2017). Metacognition in the classroom: The association between students' exam predictions and their desired grades. Consciousness and Cognition, 51, 125 – 139. https://doi.org/10.1016/j.concog.2017.03.002</bibtext> </blist> <blist> <bibtext> Serra, M. J., &amp; DeMarree, K. G. (2016). Unskilled and unaware in the classroom: College students' desired grades predict their biased grade predictions. Memory &amp; Cognition, 44, 1 – 11. https://doi.org/10.3758/s13421‐016‐0624‐9</bibtext> </blist> <blist> <bibtext> Snooks, M. K. (2004). Using practice tests on a regular basis to improve student learning. New Directions for Teaching and Learning, 2004, 109 – 113. https://doi.org/10.1002/tl.178</bibtext> </blist> <blist> <bibtext> Taylor, S. E., &amp; Brown, J. D. (1988). Illusion and well‐being: A social psychological perspective on mental health. Psychological Bulletin, 103, 193 – 210. https://doi.org/10.1037/0033‐2909.103.2.193</bibtext> </blist> <blist> <bibtext> Thiede, K. W. (1999). The importance of monitoring and self‐regulation during multitrial learning. Psychonomic Bulletin &amp; Review, 6, 662 – 667. https://doi.org/10.3758/BF03212976</bibtext> </blist> <blist> <bibtext> Thiede, K. W., Anderson, M., &amp; Therriault, D. (2003). Accuracy of metacognitive monitoring affects learning of texts. Journal of Educational Psychology, 95, 66 – 73. https://doi.org/10.1037/0022‐0663.95.1.66</bibtext> </blist> <blist> <bibtext> Tirso, R., Geraci, L. (2018). Taking another perspective on overconfidence in cognitive ability: A comparison of self and other metacognitive judgments. Manuscript under review</bibtext> </blist> </ref> <aug> <p>By Gabriel D. Saenz; Lisa Geraci and Robert Tirso</p> <p>Reported by Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib15" firstref="ref4"></nolink> <nolink nlid="nl2" bibid="bib18" firstref="ref5"></nolink> <nolink nlid="nl3" bibid="bib24" firstref="ref6"></nolink> <nolink nlid="nl4" bibid="bib28" firstref="ref7"></nolink> <nolink nlid="nl5" bibid="bib32" firstref="ref8"></nolink> <nolink nlid="nl6" bibid="bib21" firstref="ref9"></nolink> <nolink nlid="nl7" bibid="bib10" firstref="ref13"></nolink> <nolink nlid="nl8" bibid="bib17" firstref="ref20"></nolink> <nolink nlid="nl9" bibid="bib25" firstref="ref22"></nolink> <nolink nlid="nl10" bibid="bib26" firstref="ref23"></nolink> <nolink nlid="nl11" bibid="bib27" firstref="ref24"></nolink> <nolink nlid="nl12" bibid="bib20" firstref="ref26"></nolink> <nolink nlid="nl13" bibid="bib36" firstref="ref27"></nolink> <nolink nlid="nl14" bibid="bib37" firstref="ref28"></nolink> <nolink nlid="nl15" bibid="bib16" firstref="ref29"></nolink> <nolink nlid="nl16" bibid="bib29" firstref="ref33"></nolink> <nolink nlid="nl17" bibid="bib33" firstref="ref42"></nolink> <nolink nlid="nl18" bibid="bib13" firstref="ref45"></nolink> <nolink nlid="nl19" bibid="bib14" firstref="ref46"></nolink> <nolink nlid="nl20" bibid="bib19" firstref="ref47"></nolink> <nolink nlid="nl21" bibid="bib34" firstref="ref51"></nolink> <nolink nlid="nl22" bibid="bib38" firstref="ref75"></nolink> <nolink nlid="nl23" bibid="bib12" firstref="ref103"></nolink> <nolink nlid="nl24" bibid="bib11" firstref="ref104"></nolink> <nolink nlid="nl25" bibid="bib22" firstref="ref118"></nolink> <nolink nlid="nl26" bibid="bib23" firstref="ref119"></nolink> <nolink nlid="nl27" bibid="bib30" firstref="ref120"></nolink> <nolink nlid="nl28" bibid="bib31" firstref="ref121"></nolink> <nolink nlid="nl29" bibid="bib35" firstref="ref130"></nolink> |
|---|---|
| Header | DbId: eric DbLabel: ERIC An: EJ1262144 AccessLevel: 3 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Improving Metacognition: A Comparison of Interventions – Name: Language Label: Language Group: Lang Data: English – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Saenz%2C+Gabriel+D%2E%22">Saenz, Gabriel D.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-0198-5449">0000-0003-0198-5449</externalLink>)<br /><searchLink fieldCode="AR" term="%22Geraci%2C+Lisa%22">Geraci, Lisa</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-9302-2871">0000-0001-9302-2871</externalLink>)<br /><searchLink fieldCode="AR" term="%22Tirso%2C+Robert%22">Tirso, Robert</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-0407-4181">0000-0003-0407-4181</externalLink>) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="SO" term="%22Applied+Cognitive+Psychology%22"><i>Applied Cognitive Psychology</i></searchLink>. Sep-Oct 2019 33(5):918-929. – Name: Avail Label: Availability Group: Avail Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us – Name: PeerReviewed Label: Peer Reviewed Group: SrcInfo Data: Y – Name: Pages Label: Page Count Group: Src Data: 12 – Name: DatePubCY Label: Publication Date Group: Date Data: 2019 – Name: TypeDocument Label: Document Type Group: TypDoc Data: Journal Articles<br />Reports - Research – Name: Subject Label: Descriptors Group: Su Data: <searchLink fieldCode="DE" term="%22Metacognition%22">Metacognition</searchLink><br /><searchLink fieldCode="DE" term="%22Intervention%22">Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22Prediction%22">Prediction</searchLink><br /><searchLink fieldCode="DE" term="%22Accuracy%22">Accuracy</searchLink><br /><searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Motivation%22">Motivation</searchLink><br /><searchLink fieldCode="DE" term="%22Incentives%22">Incentives</searchLink><br /><searchLink fieldCode="DE" term="%22Reflection%22">Reflection</searchLink> – Name: DOI Label: DOI Group: ID Data: 10.1002/acp.3556 – Name: ISSN Label: ISSN Group: ISSN Data: 0888-4080 – Name: Abstract Label: Abstract Group: Ab Data: Accurate knowledge monitoring is critical to the learning process, as it allows one to regulate studying and test preparation. Thus, a number of investigations have attempted to improve metacognition in the classroom, with the ultimate goal of improving student exam performance. However, such interventions have had inconsistent success using varying paradigms. We compared the effectiveness of five interventions aimed at improving prediction accuracy in a laboratory environment: review, salient feedback, motivation warning lecture, incentives, and reflection. Only the salient feedback and the motivation warning lecture interventions significantly improved participants' prediction accuracy from test 1 to test 2. Review, incentives, and reflection did not improve predictive or postdictive calibration. Well-timed salient feedback and a lecture warning students not to be biased by desired grades were effective methods of improving calibration accuracy. Results offer effective interventions to improve metacognition that could be used in a classroom setting. – Name: AbstractInfo Label: Abstractor Group: Ab Data: As Provided – Name: DateEntry Label: Entry Date Group: Date Data: 2020 – Name: AN Label: Accession Number Group: ID Data: EJ1262144 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1262144 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1002/acp.3556 Languages: – Text: English PhysicalDescription: Pagination: PageCount: 12 StartPage: 918 Subjects: – SubjectFull: Metacognition Type: general – SubjectFull: Intervention Type: general – SubjectFull: Prediction Type: general – SubjectFull: Accuracy Type: general – SubjectFull: Feedback (Response) Type: general – SubjectFull: Motivation Type: general – SubjectFull: Incentives Type: general – SubjectFull: Reflection Type: general Titles: – TitleFull: Improving Metacognition: A Comparison of Interventions Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Saenz, Gabriel D. – PersonEntity: Name: NameFull: Geraci, Lisa – PersonEntity: Name: NameFull: Tirso, Robert IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 01 Type: published Y: 2019 Identifiers: – Type: issn-print Value: 0888-4080 Numbering: – Type: volume Value: 33 – Type: issue Value: 5 Titles: – TitleFull: Applied Cognitive Psychology Type: main |
| ResultId | 1 |
