View in EDS HTML Full Text PDF Full Text

Mathematics Presentation Matters: How Superfluous Brackets and Higher-Order Operator Position in Mathematics Can Impact Arithmetic Performance

Saved in:
Bibliographic Details
Title: Mathematics Presentation Matters: How Superfluous Brackets and Higher-Order Operator Position in Mathematics Can Impact Arithmetic Performance
Language: English
Authors: Alena Egorova, Vy Ngo, Allison S. Liu (ORCID 0000-0003-1075-2575), Molly Mahoney, Justine Moy, Erin Ottmar (ORCID 0000-0002-9487-7967)
Source: Mind, Brain, and Education. 2024 18(3):258-269.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 12
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Undergraduate Students, STEM Education, Mathematical Formulas, Mathematics Instruction, Eye Movements, Arithmetic, Computation, Thinking Skills, Symbols (Mathematics), Interference (Learning), Mathematics Achievement, Learning Processes
DOI: 10.1111/mbe.12421
ISSN: 1751-2271
1751-228X
Abstract: Perceptual learning theory suggests that perceptual grouping in mathematical expressions can direct students' attention toward specific parts of problems, thus impacting their mathematical reasoning. Using in-lab eye tracking and a sample of 85 undergraduates from a STEM-focused university, we investigated how higher-order operator position (HOO; i.e., multiplication/division operators and the presence of superfluous brackets impacted students' time to first fixation to the HOO, response time, and percent of correct responses). Students solved order-of-operations problems presented in six ways (3 HOO positions × presence of brackets). We found that HOO position and presence of superfluous brackets had separate and combined impacts on calculating arithmetic expressions. Superfluous brackets most influenced undergraduates' performance when higher-order operators were located in the center of mathematical expressions. Implications for learning and future directions are discussed about observing eye movements and gaining insights into students' processes when solving arithmetic expressions.
Abstractor: As Provided
Notes: https://osf.io/fbwxy/?view_only=2eb85206fc604680afc18adeea9d9440
Entry Date: 2024
Accession Number: EJ1437318
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFmQwOQdguPBpJwioQnyiR8AAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDJ4-07lHQFCsoepkeAIBEICBmjdXuCmasP7JsdQ0k3IlhXF8wE-PMpMqtLI-xl25Ep_5iZ51DQoA9_1Kuid12Z4f2ShsVexcUotVUQorj7MmS9-szdRJt50GtKe-Acfbgtmz_jMvD09xOcuwM9xfvO0hbeE9JATe4lVhAKkBdbVLKDZO_bdguCyjiIXxtSrlXcjJIeW0xw01ibhQSId95zSTpr_3EEATKHGEi5Q=
Text:
  Availability: 1
  Value: <anid>AN0179320539;[309x]01aug.24;2024Sep02.01:58;v2.2.500</anid> <title id="AN0179320539-1">Mathematics Presentation Matters: How Superfluous Brackets and Higher‐order Operator Position in Mathematics Can Impact Arithmetic Performance </title> <p>Perceptual learning theory suggests that perceptual grouping in mathematical expressions can direct students' attention toward specific parts of problems, thus impacting their mathematical reasoning. Using in‐lab eye tracking and a sample of 85 undergraduates from a STEM‐focused university, we investigated how higher‐order operator position (HOO; i.e., multiplication/division operators and the presence of superfluous brackets impacted students' time to first fixation to the HOO, response time, and percent of correct responses). Students solved order‐of‐operations problems presented in six ways (3 HOO positions × presence of brackets). We found that HOO position and presence of superfluous brackets had separate and combined impacts on calculating arithmetic expressions. Superfluous brackets most influenced undergraduates' performance when higher‐order operators were located in the center of mathematical expressions. Implications for learning and future directions are discussed about observing eye movements and gaining insights into students' processes when solving arithmetic expressions.</p> <p>Using eye tracking, we examined how the position of higher‐order operators (i.e., multiplication and division signs) and the presence of superfluous brackets (i.e., brackets that do not change the order of calculations when included) impacted undergraduate students' attentional processing and performance when solving arithmetic expressions. The results show that including superfluous brackets was most impactful when higher‐order operators were in the center of mathematical expressions. Findings indicate how subtle changes in mathematical expressions can influence arithmetic performance.</p> <p>Research in cognitive science has demonstrated that experts and struggling students alike are impacted by subtle variations in perceptual cues when solving mathematical expressions (Harrison, Smith, Hulse, & Ottmar, [<reflink idref="bib7" id="ref1">7</reflink>]; Kellman & Massey, [<reflink idref="bib10" id="ref2">10</reflink>]; Kirshner & Awtry, [<reflink idref="bib13" id="ref3">13</reflink>]; Landy & Goldstone, [<reflink idref="bib14" id="ref4">14</reflink>]; Marghetis, Landy, & Goldstone, [<reflink idref="bib18" id="ref5">18</reflink>]). Perceptual learning theory suggests that mathematics learning retrains learners' perceptual systems to attend to important cues in mathematical notation, creating new perceptual‐motor routines that align with their learned mathematical principles and rules (Goldstone, Marghetis, Weitnauer, Ottmar, & Landy, [<reflink idref="bib4" id="ref6">4</reflink>]; Marghetis et al., [<reflink idref="bib18" id="ref7">18</reflink>]); intentionally utilizing perceptual cues within mathematical expressions may help to create perceptual‐motor routines that support students as they apply mathematical principles and rules to solve mathematical expressions. Though prior studies have found that perceptual cues can positively influence mathematical problem‐solving behavior (Hoch & Dreyfus, [<reflink idref="bib9" id="ref8">9</reflink>]; Marchini & Papadopoulos, [<reflink idref="bib17" id="ref9">17</reflink>]; Ngo, Lacera, Closser, & Ottmar, [<reflink idref="bib20" id="ref10">20</reflink>]), few studies have directly studied how small variations in these perceptual cues shift students' perceptual‐motor routines.</p> <p>To help answer this question, we explore the relationship between perceptual cues and eye gaze when students solve mathematical expressions, using superfluous brackets as our visual perceptual cue. In mathematical expressions, the hierarchy of calculations can be conveyed in several ways. For example, the operators themselves can indicate the order in which calculations are to be done based on the rules of order of operations, with higher‐order operators (i.e., multiplication and division operators) taking precedence over lower‐order ones (i.e., addition and subtraction operators). Brackets are attentional cues that can also pair groups of numbers and symbols together. Sometimes, brackets alter the conventional hierarchy communicated by order of operations rules and specify the order in which operations are to be performed (e.g., (<emph>a</emph> + <emph>b</emph>) * (<emph>c</emph> + <emph>d</emph>)). Other times brackets are <emph>superfluous</emph> (Papadopoulos & Gunnarsson, [<reflink idref="bib23" id="ref11">23</reflink>]), acting as a redundant perceptual cue: the brackets perceptually group mathematical terms together, but removing the brackets would result in the same order of calculations (e.g., (<emph>a</emph>*<emph>b</emph>) + (<emph>c</emph>*<emph>d</emph>)). Here, we investigate whether perceptual cues (that are otherwise mathematically redundant) can direct students' attention beyond what is communicated by order of operations rules; specifically, we investigate how the position of higher‐order operators (HOO) and the presence of superfluous brackets separately and jointly impact college‐aged students' attention (measured using eye tracking measures) and performance (measured using response time and percent of correct responses) when solving arithmetic expressions.</p> <hd id="AN0179320539-2">The Utility of Superfluous Brackets in Arithmetic Expressions</hd> <p>Many students mentally apply brackets when interpreting algebraic expressions, even when brackets are not physically present (Papadopoulos & Gunnarsson, [<reflink idref="bib23" id="ref12">23</reflink>]; Papadopoulos & Thoma, [<reflink idref="bib24" id="ref13">24</reflink>]). Building off this natural inclination, several studies have investigated how superfluous brackets influence mathematical performance (Hoch & Dreyfus, [<reflink idref="bib9" id="ref14">9</reflink>]; Marchini & Papadopoulos, [<reflink idref="bib17" id="ref15">17</reflink>]). Linchevski and Livneh ([<reflink idref="bib16" id="ref16">16</reflink>]) suggested that students who lacked structure sense—the ability to understand mathematical structures—would struggle with arithmetic expressions of the type <emph>a</emph> + <emph>b</emph> * <emph>c</emph>, as students would need to understand that <emph>b</emph> should be combined with <emph>c</emph>, not <emph>a</emph>, due to order of precedence; they suggested that inserting superfluous brackets around <emph>b</emph> * <emph>c</emph> could make the order of operations rules more salient, and emphasize that the multiplication operation should be calculated first. Indeed, studies have found that students are more accurate at calculating equations that included superfluous brackets to highlight important arithmetical structures compared to equations that did not (Hoch & Dreyfus, [<reflink idref="bib9" id="ref17">9</reflink>]; Marchini & Papadopoulos, [<reflink idref="bib17" id="ref18">17</reflink>]). However, superfluous brackets have also been found to have no effect (Gunnarsson, Sönnerhed, & Hernell, [<reflink idref="bib6" id="ref19">6</reflink>]) or even lead to misinterpretations and procedural errors (Ayres, [<reflink idref="bib1" id="ref20">1</reflink>]; Hewitt, [<reflink idref="bib8" id="ref21">8</reflink>]; Kieran, [<reflink idref="bib11" id="ref22">11</reflink>]). Thus, in addition to understanding superfluous brackets' influences on calculational outcomes, it is important to understand <emph>how</emph> superfluous brackets influence students' attentional and calculational processes to recognize the situations where they may help versus hinder students.</p> <hd id="AN0179320539-3">Using Eye Tracking to Measure Mathematical Processes</hd> <p>Researchers have used eye tracking in several mathematical domains to investigate cognitive processing during mathematical problem‐solving (see Strohmaier, MacKay, Obersteiner, & Reiss, [<reflink idref="bib27" id="ref23">27</reflink>]). Though eye tracking is not an exhaustive measure of attention, and attention does not always coincide directly with eye fixations (Schindler & Lilienthal, [<reflink idref="bib25" id="ref24">25</reflink>]), these methods can still indicate areas of interest during cognitive processing. If eye fixations can indicate where and when students attend to parts of arithmetic expressions, then they can also measure how HOO position and the presence of superfluous brackets influence students' eye gaze and attention when solving arithmetic expressions.</p> <p>Eye tracking studies in mathematics education have separately investigated operators and brackets. Landy, Jones, and Goldstone ([<reflink idref="bib15" id="ref25">15</reflink>]) found that undergraduates with prior algebra knowledge showed earlier and longer eye fixations to multiplication signs than plus signs in the context of mathematical expressions, suggesting that undergraduates selectively attend to higher‐order operators over lower‐order ones. Schneider, Muruyama, Dehaene, and Sigman ([<reflink idref="bib26" id="ref26">26</reflink>]) also found that adults' fixations corresponded to the hierarchical structure of expressions, regardless of whether that structure was communicated by operator precedence or brackets, and that there was a left‐right bias in adults' fixation patterns that needed to be corrected when it conflicted with the expression's structure. Open questions remain about the nature of eye movements or fixations when solving arithmetic expressions that involve operator precedence and superfluous brackets together, which can inform how order of operations rules and superfluous perceptual cues may interact to influence student processes and arithmetic performance.</p> <hd id="AN0179320539-4">The Current Study</hd> <p>In previous work (Ngo et al., [<reflink idref="bib20" id="ref27">20</reflink>]), our team used a between‐subjects experiment to behaviorally investigate how HOO position and superfluous brackets impacted middle school students' arithmetic performance and response time when simplifying expressions. Students were most accurate when the HOO was on the left (e.g., 5*2 + 3 + 4), least accurate when the HOO was on the right (e.g., 5 + 2 + 3*4), and more efficient when solving problem sets with superfluous brackets compared to problem sets without superfluous brackets (e.g., (5*2) + 3 + 4 vs. 5*2 + 3 + 4). Moderation analyses revealed that these effects were largely driven by the center position: when the HOO was in the center (compared to the left), students were more accurate when solving expressions with brackets but less accurate without brackets. These findings suggest that the benefit of superfluous brackets may depend on HOO position, but it is unclear how HOO position and superfluous brackets relate to cognitive and attentional processes that may influence how students solve arithmetic expressions.</p> <p>Here, we use: (<reflink idref="bib1" id="ref28">1</reflink>) a within‐subjects design to build on our prior between‐subject findings (Ngo et al., [<reflink idref="bib20" id="ref29">20</reflink>]) and (<reflink idref="bib2" id="ref30">2</reflink>) eye tracking methods to better understand how HOO position and the presence of superfluous brackets influence students' time to first fixation to the HOO—a critical element when solving mathematics problems involving order of operations rules—in addition to performance outcomes, while solving simple arithmetic expressions. To explore whether higher baseline mathematics skills would significantly change the results found in our prior study (Ngo et al., [<reflink idref="bib20" id="ref31">20</reflink>]), we recruited undergraduates from a STEM‐focused university whose arithmetic knowledge and consequent perceptual learning should be more advanced than the middle school sample.</p> <p>The study hypotheses, design, and data analysis plan were preregistered on Open Science Framework (OSF) at https://osf.io/fbwxy/?view%5fonly=2eb85206fc604680afc18adeea9d9440. We investigated:</p> <hd id="AN0179320539-5">RQ1</hd> <p>Does HOO position within an arithmetic expression affect attentional processing, as measured by time to first fixation to the HOO, and arithmetic performance, as measured by response time and percent of correct responses?</p> <p>Based on previous findings showing that students have strong tendencies to calculate from left to right even when it conflicts with the order of operations (Banerjee & Subramaniam, [<reflink idref="bib2" id="ref32">2</reflink>]; Blando, Kelly, Schneider, & Sleeman, [<reflink idref="bib3" id="ref33">3</reflink>]; Gunnarsson et al., [<reflink idref="bib6" id="ref34">6</reflink>]), we expected participants to similarly demonstrate a left‐to‐right tendency when solving arithmetic expressions and automatically focus on the leftmost operator first. This tendency should benefit arithmetic performance when the leftmost operator is the HOO and meant to be calculated first. Thus, solving expressions with the HOO on the left would lead to (<reflink idref="bib1" id="ref35">1</reflink>) faster times to first fixation to the HOO, (<reflink idref="bib2" id="ref36">2</reflink>) shorter response times to solve the expressions, and (<reflink idref="bib3" id="ref37">3</reflink>) higher percent of correct responses compared to solving expressions with the HOO in the center and on the right.</p> <hd id="AN0179320539-6">RQ2</hd> <p>Does the presence of superfluous brackets within an arithmetic expression impact attentional processing and arithmetic performance?</p> <p>In line with perceptual learning theory (Goldstone et al., [<reflink idref="bib4" id="ref38">4</reflink>]; Marghetis et al., [<reflink idref="bib18" id="ref39">18</reflink>]), we hypothesized that superfluous brackets would draw participants' attention to the HOO (even though the superfluous brackets only add mathematically redundant information), making it easier for students to quickly comprehend the correct calculational order. Thus, solving expressions with superfluous brackets would lead to (<reflink idref="bib1" id="ref40">1</reflink>) faster times to first fixation to the HOO, (<reflink idref="bib2" id="ref41">2</reflink>) shorter response times, and (<reflink idref="bib3" id="ref42">3</reflink>) higher percent of correct responses compared to solving expressions without superfluous brackets.</p> <hd id="AN0179320539-7">RQ3</hd> <p>Is there an interaction between HOO position and the presence of superfluous brackets on attentional processing and arithmetic performance?</p> <p>We hypothesized that superfluous brackets would modify the effects of HOO position on attentional processing and arithmetic performance. As stated above, we expect participants to normally exhibit a left‐to‐right tendency such that left HOOs are attended to more quickly (RQ1's hypothesis), and we also expect expressions with superfluous brackets to be attended to and answered more quickly compared to expressions without superfluous brackets, as the brackets should serve as attentional cues toward the HOO (RQ2's hypothesis). Therefore, superfluous brackets should reduce the time to first fixation, reduce response time, and increase accuracy most strongly for middle and right HOOs that would benefit the most from the redundant attentional cue, as these HOO positions do not align with students' tendency to calculate from left‐to‐right. Thus, differences between first fixation to the HOO, response times, and percent of correct responses should be reduced for center and right HOOs (compared to left HOOs) for arithmetic expressions with brackets compared to expressions without brackets.</p> <hd id="AN0179320539-8">METHODS</hd> <p></p> <hd id="AN0179320539-9">Participants</hd> <p>Eighty‐five undergraduate students (demographics in Table 1) from a STEM‐focused university completed the study during the 2021–2022 school year for psychology class research credits (students did not have to major in psychology to participate). Students likely held at least average or above prior mathematics knowledge, as the university's admissions requirements include a high average GPA, 4 years of mathematics courses, and a course load with a strong emphasis on mathematics and science. Participation was voluntary. The study received approval from the Institutional Review Board. All data, analysis code, and research materials are available on the OSF project page.</p> <p>1 Table Student Participant Demographics in the Current Study (N  = 85).</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Gender</th><th align="center">%</th></tr></thead><tbody valign="top"><tr><td align="left">Female</td><td align="left" char=".">72.6</td></tr><tr><td align="left">Male</td><td align="left" char=".">22.6</td></tr><tr><td align="left">Gender Variant/Non‐conforming</td><td align="left" char=".">2</td></tr><tr><td align="left">Not reported</td><td align="left" char=".">3</td></tr></tbody></table> </ephtml> </p> <p></p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Grade</th><th align="center">%</th></tr></thead><tbody valign="top"><tr><td align="left">Freshman</td><td align="char" char=".">21.6</td></tr><tr><td align="left">Sophomore</td><td align="char" char=".">33.3</td></tr><tr><td align="left">Junior</td><td align="char" char=".">14.7</td></tr><tr><td align="left">Senior</td><td align="char" char=".">23.5</td></tr><tr><td align="left">Not reported</td><td align="char" char="."> 6.9</td></tr></tbody></table> </ephtml> </p> <hd id="AN0179320539-10">Study Procedure</hd> <p>The study was designed using Tobii Pro Lab and eye tracking hardware (Tobii Pro, [<reflink idref="bib28" id="ref43">28</reflink>]). Participants individually completed a 30‐min lab session. After signing consent forms, participants viewed a nine‐dot calibration sequence on a 50 cm computer screen. Participants' eye fixations on the calibration dots were analyzed by the software to ensure accuracy of eye movement data. Specifically, Tobii Pro Lab used part of the calibration data to build a prediction model (reporting the model's accuracy and precision based on the gaze data's position relative to the stimuli from this calibration dataset), and then validated the prediction model on the remaining data (reporting the model's accuracy and precision on this validation dataset). Across participants, calibration achieved 7.7 mm accuracy and validation demonstrated 6.2 mm accuracy. A research assistant also evaluated the results for validity; if large offsets were found, participants recalibrated until no atypical offsets were present.</p> <p>After calibration, participants viewed and mentally solved arithmetic expressions (see Figure 1 for details). To avoid practice effects inherent in a within‐participants design (i.e., enhanced accuracy caused by participants getting more practice), we designed 24 distinct arithmetic expressions that involved order of operations (based on 5th grade Common Core Standards for "Operations and Algebraic Thinking"; National Governors Association Center for Best Practices, [<reflink idref="bib19" id="ref44">19</reflink>]) (see Appendix A). Arithmetic expressions were presented horizontally to stay consistent with our past work and facilitate comparison across the middle school and undergraduate populations (Ngo et al., [<reflink idref="bib20" id="ref45">20</reflink>]). There were four arithmetic expressions per condition that varied HOO position and the presence of superfluous brackets (see Figure 2). All arithmetic expressions consisted of four single‐digit numbers (1–8) and three operators: one higher‐order operator (i.e., multiplication or division) and two were lower‐order operators (i.e., addition or subtraction). Single‐digit numbers were evenly distributed across all arithmetic expressions. The digits 1–8 were chosen to help keep the number of arithmetic expressions with solution magnitudes above and below 10 relatively equal: 13 of the solutions were of magnitudes under 10, and 11 were equal to or greater than 10. Participants completed a three‐trial training to familiarize themselves with the experiment, and then solved the 24 order‐of‐operations arithmetic expressions. All participants solved the arithmetic expressions in the same order. Lastly, participants completed an optional demographics survey and were debriefed.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/309X/01aug24/mbe12421-fig-0001.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="mbe12421-fig-0001.jpg" title="1 Example of trials in Tobii Pro Lab. This sequence shows an example of how a participant progressed through the study in Tobii Pro Lab. (a) Participants first saw an instruction screen informing them of the study procedure and were instructed to press the space bar to continue. (b) Participants saw the first arithmetic expression and simplified the expression in their head; arithmetic expressions were displayed until participants wanted to answer. (c) Once they had an answer, the participant pressed the space bar, saw an instruction screen, and said their answer out loud. This procedure was implemented to: (1) record the time participants needed to calculate an answer (i.e., their response time), and (2) ensure that participants' eyes were looking at the bottom of the screen after each trial and not where expressions would appear in the next trial, which could inflate time to first fixation to the HOO. A research assistant recorded the verbal response and then informed the participant to press the space bar to continue. (d) Participants viewed and solved the next arithmetic expression." /> </p> <p></p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/309X/01aug24/mbe12421-fig-0002.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="mbe12421-fig-0002.jpg" title="2 Structure of the study's six conditions: 3 positions of HOO × 2 presence of superfluous brackets." /> </p> <p></p> <hd id="AN0179320539-13">Stimuli and Areas of Interest</hd> <p>Each arithmetic expression was presented in black 60‐point Arial font on a white background. We defined areas of interest around each number, operator, and bracket. Expressions without superfluous brackets had seven interest areas, whereas expressions with brackets had nine. Each area of interest measured 5.4 cm by 5.4 cm, centered around the stimulus component. The borders of each area of interest were connected but did not overlap (Figure 3).</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/309X/01aug24/mbe12421-fig-0003.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="mbe12421-fig-0003.jpg" title="3 Example of arithmetic expressions with the areas of interest illustrated." /> </p> <p></p> <hd id="AN0179320539-15">Measures</hd> <p></p> <hd id="AN0179320539-16">Time to First Fixation to the Higher‐order Operators</hd> <p>A fixation was defined as a period of at least 60 ms (based on the default Tobii I‐VT filter's fixation duration minimum) in which participants' eye position did not move more than a certain degree of visual angle; this threshold was set by the Tobii algorithm based on each participant's individual eye velocity (Olsen, [<reflink idref="bib21" id="ref46">21</reflink>]). We considered a fixation within a notation's area of interest to be a fixation on that specific notation.</p> <p>For each arithmetic expression, the Tobii Pro Lab software recorded in milliseconds how long students took to first fixate on the HOO, starting from the time the expression was presented on screen. A quicker time to first fixation indicated that participants attended to the operator more quickly.</p> <hd id="AN0179320539-17">Response Time</hd> <p>For each arithmetic expression, the Tobii Pro Lab system recorded in milliseconds how long students took to solve the arithmetic expression, measured as the time the arithmetic expression was presented on screen until the time the participant pressed the space bar to move to the instruction screen (i.e., the time spent on (b) in Figure 1).</p> <hd id="AN0179320539-18">Percent of Correct Responses</hd> <p>A research assistant recorded participants' verbal answers for each arithmetic expression. Responses were coded for each participant on whether the answer was correct (<reflink idref="bib1" id="ref47">1</reflink>) or not (0). We then calculated the percent of correct responses given for each arithmetic expression type across all participants, and this percent was used for analyses.</p> <hd id="AN0179320539-19">APPROACH TO ANALYSIS</hd> <p></p> <hd id="AN0179320539-20">Data Preprocessing</hd> <p>Data were analyzed using R Studio and the lme4 package. Analyses included 84 participants as one participant was excluded due to software glitches. For the time to first fixation analyses, we only included arithmetic expressions in which participants fixated on the HOO at least once, resulting in 1807 observations: 303 no brackets‐left trials, 301 brackets‐left trials, 316 no brackets‐center trials, 304 brackets‐center trials, 300 no brackets‐right trials, and 283 brackets‐right trials. For the response time and percent of correct responses analyses, we used all 2016 observations, with 336 trials in each condition.</p> <hd id="AN0179320539-21">Analyses</hd> <p>We conducted three multilevel linear models with condition (HOO position and brackets presence) at the first level (problem‐level) and participant ID at the second level (student‐level). Because we only wanted to control for variance in our three outcomes between participants and not for differences caused by HOO position and/or superfluous brackets, we assumed the predictors' effects were fixed across participants and the intercept was random.</p> <p>We conducted separate models using average time to first fixation to the HOO, average response time, and percent of correct responses as dependent variables (Table 2). First, unconditional models were estimated, and we calculated the intraclass correlation coefficient (ICC) to analyze if levels of our dependent variables varied among participants. To answer RQ1 and RQ2, we added the main predictors (HOO position and the presence of superfluous brackets) to analyze their fixed effects on each outcome. HOO position was dummy coded, using HOO on the left as the reference group. To address RQ3, we added interaction terms to our three models, which detected whether there was an interaction between HOO position and the presence of superfluous brackets on solution processes and arithmetic performance.</p> <p>2 Table Effects of Higher‐order Operators (HOO) Position and Brackets Presence on Time to First Fixation to the HOO, Response Time, and Percent of Correct Responses</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" valign="top" /><th align="center">Time to first fixation to the HOO</th><th align="center">Response time</th><th align="center">Percent of correct responses</th></tr><tr><th align="center"><italic>n</italic> = 1807 observations</th><th align="center"><italic>n</italic> = 2016 observations</th><th align="center"><italic>n</italic> = 2016 observations</th></tr><tr><th align="left">Variable</th><th align="center">Model 1</th><th align="center">Model 2</th><th align="center">Model 1</th><th align="center">Model 2</th><th align="center">Model 1</th><th align="center">Model 2</th></tr><tr><th align="left">Fixed effects</th><th align="center">B (SE)</th><th align="center"><italic>t</italic>‐value</th><th align="left">B (SE)</th><th align="center"><italic>t</italic>‐value</th><th align="center">B (SE)</th><th align="center"><italic>t</italic>‐value</th><th align="center">B (SE)</th><th align="center"><italic>t</italic>‐value</th><th align="center">B (SE)</th><th align="center"><italic>t</italic>‐value</th><th align="center">B (SE)</th><th align="center"><italic>t</italic>‐value</th></tr></thead><tbody valign="top"><tr><td align="left" valign="top">Intercept</td><td align="char" char=".">407.8***</td><td align="char" char="." valign="top">8.9</td><td align="char" char=".">396.5***</td><td align="char" char="." valign="top">7.5</td><td align="char" char=".">4,435.1***</td><td align="char" char="." valign="top">20.3</td><td align="char" char=".">4,082.1***</td><td align="char" char="." valign="top">17.9</td><td align="char" char=".">0.99***</td><td align="char" char="." valign="top">73.7</td><td align="char" char=".">0.99***</td><td align="char" char="." valign="top">61.2</td></tr><tr><td align="char" char=".">(45.7)</td><td align="char" char=".">(52.9)</td><td align="char" char=".">(218.6)</td><td align="char" char=".">(228.4)</td><td align="char" char=".">(0.01)</td><td align="char" char=".">(0.02)</td></tr><tr><td align="left" valign="top">HOO center</td><td align="char" char=".">253.6***</td><td align="char" char="." valign="top">5.3</td><td align="char" char=".">368.0***</td><td align="char" char="." valign="top">5.5</td><td align="char" char=".">658.1***</td><td align="char" char="." valign="top">5.2</td><td align="char" char=".">1,333.7***</td><td align="char" char="." valign="top">7.8</td><td align="char" char=".">−0.03</td><td align="char" char="." valign="top">−1.8</td><td align="char" char=".">−0.03</td><td align="char" char="." valign="top">−1.5</td></tr><tr><td align="char" char=".">(47.9)</td><td align="char" char=".">(66.5)</td><td align="char" char=".">(125.4)</td><td align="char" char=".">(171.5)</td><td align="char" char=".">(0.02)</td><td align="char" char=".">(0.02)</td></tr><tr><td align="left" valign="top">HOO right</td><td align="char" char=".">797.9***</td><td align="char" char="." valign="top">16.7</td><td align="char" char=".">715.7***</td><td align="char" char="." valign="top">10.7</td><td align="char" char=".">1,263.1***</td><td align="char" char="." valign="top">10.1</td><td align="char" char=".">1,646.3***</td><td align="char" char="." valign="top">9.6</td><td align="char" char=".">−0.19***</td><td align="char" char="." valign="top">−12.2</td><td align="char" char=".">−0.18***</td><td align="char" char="." valign="top">−8.2</td></tr><tr><td align="char" char=".">(47.9)</td><td align="char" char=".">(66.7)</td><td align="char" char=".">(125.4)</td><td align="char" char=".">(171.5)</td><td align="char" char=".">(0.02)</td><td align="char" char=".">(0.02)</td></tr><tr><td align="left" valign="top">Without brackets</td><td align="char" char=".">29.9</td><td align="char" char="." valign="top">0.8</td><td align="char" char=".">52.2</td><td align="char" char="." valign="top">0.8</td><td align="char" char=".">270.1**</td><td align="char" char="." valign="top">2.7</td><td align="char" char=".">976.1***</td><td align="char" char="." valign="top">5.7</td><td align="char" char=".">−0.01</td><td align="char" char="." valign="top">−0.9</td><td align="char" char=".">−0.01</td><td align="char" char="." valign="top">−0.4</td></tr><tr><td align="char" char=".">(39.1)</td><td align="char" char=".">(66.7)</td><td align="char" char=".">(102.4)</td><td align="char" char=".">(171.5)</td><td align="char" char=".">(0.01)</td><td align="char" char=".">(0.02)</td></tr><tr><td align="left" valign="top">HOO center × Without brackets</td><td align="char" char=".">–</td><td align="char" char="." valign="top">–</td><td align="char" char=".">−229.9*</td><td align="char" char="." valign="top">−2.4</td><td align="char" char="." valign="top">–</td><td align="char" char="." valign="top">–</td><td align="char" char=".">−1,351.2***</td><td align="char" char="." valign="top">−5.6</td><td align="char" char="." valign="top">–</td><td align="char" char="." valign="top">–</td><td align="char" char=".">0.01</td><td align="char" char="." valign="top">0.3</td></tr><tr><td align="left" /><td align="char" char=".">(94.0)</td><td align="char" char=".">(242.5)</td><td align="char" char=".">(0.03)</td></tr><tr><td align="left" valign="top">HOO right × Without brackets</td><td align="char" char=".">–</td><td align="char" char="." valign="top">–</td><td align="char" char=".">163.5</td><td align="char" char="." valign="top">1.7</td><td align="char" char="." valign="top">–</td><td align="char" char="." valign="top">–</td><td align="char" char=".">−766.6**</td><td align="char" char="." valign="top">−3.2</td><td align="char" char="." valign="top">–</td><td align="char" char="." valign="top">–</td><td align="char" char=".">−0.02</td><td align="char" char="." valign="top">−0.6</td></tr><tr><td align="left" /><td align="char" char=".">−94.0</td><td align="char" char=".">−242.5</td><td align="char" char=".">−0.03</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note</emph>. Left HOO and brackets were used as the reference group for HOO and no bracket comparisons, respectively. *<emph>p</emph> < .05, **<emph>p</emph> < .01, ***<emph>p <</emph> .001.</p> <hd id="AN0179320539-22">RESULTS</hd> <p></p> <hd id="AN0179320539-23">Intraclass Correlation Coefficients (ICCs)</hd> <p>ICCs for time to first fixation to the HOO (0.07) and response time (0.65) were greater than 0.05, indicating that outcomes varied among participants and multilevel modeling (MLM) was necessary to account for the nesting of multiple arithmetic expression types per participant. Although the ICC for percent of correct responses was low (0.00), we used MLM on this variable as well to ensure that results were not affected by our nested within‐participants experimental design.</p> <hd id="AN0179320539-24">RQ1: Impact of Higher Order Operator Position</hd> <p>We hypothesized that solving expressions with the HOO on the left would lead to faster time to first fixation, shorter response times, and higher percent of correct responses compared to expressions with the HOO in the center and on the right, because the HOO position would be consistent with students' natural left‐to‐right calculational tendencies. In accordance with our hypothesis, HOO position significantly predicted all three outcomes: students showed faster time to first fixation (<emph>p</emph> < .001), shorter response times (<emph>p</emph> < .001), and higher percent of correct responses (<emph>p</emph> < .001) for arithmetic expressions with the HOO on the left compared to the center and/or the right. When the HOO was on the left, students fixated on the HOO more quickly than when solving arithmetic expressions with the HOO in the center or on the right (Figure 4a). Compared to left HOOs, students took approximately 250 ms longer to fixate on center HOOs and approximately 800 ms longer to fixate on right HOOs. Students also answered more quickly when the HOO was on the left than when the HOO was in the center or on the right, taking approximately 620 ms longer when the HOO was in the center and approximately 1,250 ms longer when it was on the right compared to the left (Figure 4b). Finally, students were approximately 20% more accurate when the HOO was on the left than when the HOO was on the right (Figure 4c).</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/309X/01aug24/mbe12421-fig-0004.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="mbe12421-fig-0004.jpg" title="4 Students' average (a) Time to first fixation to the higher‐order operators (ms), (b) Response time (ms), and (c) Percent of correct responses based on HOO position." /> </p> <p></p> <hd id="AN0179320539-26">RQ2: Impact of Presence of Brackets</hd> <p>We expected that solving expressions with superfluous brackets would lead to faster time to first fixation, shorter response times, and higher percent of correct responses compared to solving expressions without superfluous brackets because the brackets can quickly reiterate which operations must be calculated first. This hypothesis was partially supported: although brackets around the HOO were related to shorter response times (<emph>p</emph> < .01) (Figure 5a), no significant relations were found with time to first fixation to the HOO or percent of correct responses (Figure 5b,c).</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/309X/01aug24/mbe12421-fig-0005.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="mbe12421-fig-0005.jpg" title="5 Students' average (a) Time to first fixation to the higher‐order operators (ms), (b) Response time (ms), and (c) Percent of correct responses with superfluous brackets versus no superfluous brackets." /> </p> <p></p> <hd id="AN0179320539-28">RQ3: Interaction of HOO Position and Presence of Brackets</hd> <p>We hypothesized that: (<reflink idref="bib1" id="ref48">1</reflink>) time to first fixation, response times, and percent of correct responses would be best when the HOO was on the left and brackets were present, as both the HOO and brackets would align with students' natural left‐to‐right calculational tendency; and (<reflink idref="bib2" id="ref49">2</reflink>) the presence of brackets would attenuate the difference between left HOOs and center/right HOOs because the brackets would help direct student attention to the operators that should be calculated first. Results only partially supported our hypotheses. There were significant interactions between HOO position and the presence of brackets for time to first fixation (Figure 6a) and response time (Figure 6b), but not for percent of correct responses. In arithmetic expressions with and without brackets, students fixated faster on left HOOs compared to center HOOs. However, this difference was actually significantly <emph>larger</emph> in arithmetic expressions with superfluous brackets (Figure 6a): the addition of superfluous brackets did not appear to affect fixation time when the HOO was on the left, but participants took approximately 200 ms longer to fixate on center HOOs with superfluous brackets compared to center HOOs without brackets. No interaction with brackets was found between arithmetic expressions with left or right HOOs: in arithmetic expressions with and without brackets, students showed faster time to first fixation when the HOO was on the left compared to the right.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/309X/01aug24/mbe12421-fig-0006.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="mbe12421-fig-0006.jpg" title="6 Students' average (a) Time to first fixation to the higher‐order operators (ms) and (b) Response time (ms) based on HOO position and presence of superfluous brackets." /> </p> <p></p> <p>As expected, response times were faster when the HOO was on the left compared to the center for arithmetic expressions with brackets. Unexpectedly, no difference in response time was found between left and center HOOs in arithmetic expressions without brackets (<emph>p</emph> < .001) (Figure 6b). Moreover, response times were significantly faster when the HOO was on the left compared to the right when brackets were present, but not when brackets were not present (<emph>p</emph> < .01).</p> <hd id="AN0179320539-30">DISCUSSION</hd> <p>The current study explored whether HOO position and the presence of superfluous brackets would separately and jointly impact undergraduates' time to first fixation to the HOO, response time, and percent of correct responses. There were three main findings. First, undergraduates were quicker to fixate on the HOO, quicker to respond, and more accurate when solving arithmetic expressions where the HOO was on the left compared to either center or right HOOs. Second, undergraduates were quicker to respond to arithmetic expressions with superfluous brackets compared to arithmetic expressions without brackets. Third, the difference in time to first fixation on the HOO on the left compared to the center was larger when superfluous brackets were present, and undergraduates were quicker to respond to arithmetic expressions with the HOO on the left compared to the center only when superfluous brackets were present. Together, these results suggest that superfluous brackets as a perceptual cue may help to emphasize information above what is conveyed by operator precedence alone, but its influence is the strongest in combination with HOO position—specifically, when the HOO is in the center.</p> <hd id="AN0179320539-31">Students Were Quicker and More Accurate at Solving Arithmetic Expressions with Left HOOs</hd> <p>Our findings about HOO position align with prior studies showing that students tend to calculate from left to right, regardless of order of operations (Banerjee & Subramaniam, [<reflink idref="bib2" id="ref50">2</reflink>]; Blando et al., [<reflink idref="bib3" id="ref51">3</reflink>]; Gunnarsson et al., [<reflink idref="bib6" id="ref52">6</reflink>]). We show that this left‐to‐right tendency is reflected not only in students' arithmetic performance, but also in students' online attentional processes. Notably, HOO position did not significantly affect percent of correct responses when the HOO was in the center compared to the left as we had expected; this finding may suggest that HOO position in isolation may have the most impact on attentional and speed related metrics (reflected by time to first fixation and response times), but not necessarily translate into higher percent of correct responses.</p> <hd id="AN0179320539-32">Students Responded More Quickly to Arithmetic Expressions with Superfluous Brackets</hd> <p>Our results suggest that brackets may have drawn students' attention toward relevant mathematical structures (in this case, the terms that they should calculate first based on order of operations), facilitating more efficient processing and solving of arithmetic expressions (Linchevski & Livneh, [<reflink idref="bib16" id="ref53">16</reflink>]). However, the lack of a time to first fixation effect contradicts prior eye tracking studies where brackets helped adults fixate to hierarchical structures (Schneider et al., [<reflink idref="bib26" id="ref54">26</reflink>]). Our finding may indicate that superfluous brackets help students attend generally to the terms that should be calculated first, but not immediately to the HOO, providing a nuanced understanding about the order in which students may process arithmetic expressions. There was also no difference in percent of correct responses for arithmetic expressions with and without superfluous brackets. Prior work has been inconsistent regarding the benefits of superfluous brackets (Gunnarsson et al., [<reflink idref="bib6" id="ref55">6</reflink>]; Hoch & Dreyfus, [<reflink idref="bib9" id="ref56">9</reflink>]; Marchini & Papadopoulos, [<reflink idref="bib17" id="ref57">17</reflink>]; Papadopoulos & Gunnarsson, [<reflink idref="bib22" id="ref58">22</reflink>], [<reflink idref="bib23" id="ref59">23</reflink>]); our work shows that the impact of brackets is not uniform across all arithmetic expression structures without considering HOO position.</p> <hd id="AN0179320539-33">HOO Position Moderated the Impact of Brackets</hd> <p>Indeed, we found that the addition of superfluous brackets appeared to amplify the congruency effect between left and center HOOs, leading to faster responses with left HOOs (where HOO position is congruent with a left‐right tendency) and slower fixation times with center HOOs (where HOO position is incongruent with a left‐right tendency). However, percent of correct responses between those conditions did not differ, implying that participants still performed calculations in the right order. Taken together, these findings of different speeds but equal accuracies for center HOOs could indicate that when arithmetic expressions do not have brackets, students had to scan all operators first to determine where to start calculating; however, in arithmetic expressions with brackets, students may have been able to start calculating before looking at the HOO itself, as the brackets already showed them that they should begin calculating the terms within the brackets. Brackets' larger influence on center HOOs is also in accordance with our prior work with middle school students (Ngo et al., [<reflink idref="bib20" id="ref60">20</reflink>]), in which superfluous brackets <emph>improved</emph> accuracy when students solved expressions with center HOOs compared to left HOOs. Similarly, the lower response time difference between center and right HOOs seen in expressions with brackets could indicate that brackets are helping to draw immediate attention to the right side terms, allowing students to begin calculations even if they do not fixate specifically to the HOO.</p> <p>If brackets do not direct attention specifically to HOOs but to the overall parenthetical term, brackets could be more helpful in terms of accuracy for arithmetic expressions in which parenthetical terms are treated as whole terms (e.g., arithmetic expressions with fractions in which whole groupings can be canceled to solve the expression), as opposed to the current study's arithmetic expressions where the superfluous brackets only indicated which terms should be calculated first. Future studies can investigate whether superfluous brackets are more helpful (or confusing) on complex algebraic expressions that cannot be easily mentally calculated.</p> <p>The response time findings suggest that the interaction between HOO position and superfluous brackets is primarily driven by a decrease in response time when the HOO is on the left and superfluous brackets are included, compared to when brackets are not present; this could reflect an additive effect of HOO position and brackets when the HOO is in the leftmost position. We note that brackets around HOOs on the left and HOOs in the center overlap (e.g., the brackets for (<emph>a</emph> * <emph>b</emph>) + <emph>c</emph> + <emph>d</emph> and <emph>a</emph> + (<emph>b</emph> * <emph>c</emph>) + <emph>d</emph> share the <emph>b</emph> term), and the brackets emphasize whether the shared <emph>b</emph> term is grouped with the <emph>a</emph> or <emph>c</emph> term. Thus, superfluous brackets may be more impactful when the grouping of terms becomes ambiguous without brackets, explaining the presence of an effect for only center HOOs and not right HOOs. Future studies can further investigate these possible explanations for why brackets have a greater effect when the HOO is in the center.</p> <hd id="AN0179320539-34">Limitations and Future Directions</hd> <p>The current study has several limitations. First, students who did not fixate on the HOO at least once when solving arithmetic expressions were filtered out of the time to first fixation analyses. Technological errors may have caused the lack of fixations (e.g., fixations were too fast for the software to detect). Alternatively, prior work has found that learners can still attend to parts of an expression even if they do not directly fixate on it (Schindler & Lilienthal, [<reflink idref="bib25" id="ref61">25</reflink>]); participants may have used other visual processes such as peripheral vision. Notably, most participants who did not fixate on the HOO still answered the arithmetic expressions correctly. As suggested earlier, at least in arithmetic expressions with superfluous brackets, the brackets may have provided enough information about calculational order for students to solve the arithmetic expressions without needing to fixate on the HOO itself. An open question is whether meaningful differences exist between arithmetic expressions where students used direct versus peripheral visual processing. Investigating whether these trials differ in attentional processes or calculational outcomes and what predicts whether a fixation occurred may provide additional information about how students solve arithmetic expressions based on their fixation patterns.</p> <p>Second, our study investigated a limited range of eye tracking and arithmetic performance outcomes. For eye tracking, we only included time to first fixation and lacked trial randomization controls for eye wandering; thus, time to first fixations may have been artificially inflated based on learned perceptual patterns across arithmetic expressions. For arithmetic performance, we looked at percent of correct responses on simple mental arithmetic expressions with multiple operators. However, other studies have found varying effects with other accuracy measures. For example, superfluous brackets positively impacted accuracy when measured as the number of problems it took for participants to solve three arithmetic expressions in a row correctly (Ngo et al., [<reflink idref="bib20" id="ref62">20</reflink>]) or as success rate on addition and subtraction equations (Marchini & Papadopoulos, [<reflink idref="bib17" id="ref63">17</reflink>]). In contrast, superfluous brackets had no effect on post‐test accuracy after an instructional intervention (Gunnarsson et al., [<reflink idref="bib6" id="ref64">6</reflink>]). Including a variety of eye tracking and accuracy measures can confirm that our fixation speed effects are robust across designs, strengthen findings on how perceptual grouping may impact underlying cognitive processes, and provide a full understanding of how perceptual cues, solution processes, and arithmetic performance relate during algebraic thinking. Future studies can also include measures that examine potential mechanisms such as inhibition skills, in addition to varying HOO position and presence of brackets. This may determine whether superfluous brackets help to inhibit participants' left‐to‐right calculational heuristics when the HOO is in the center compared to the left.</p> <p>Finally, we acknowledge a general lack of effects in percent of correct responses in several of our analyses. Although the percent of correct responses results in combination with the time to first fixation and response time findings still provide information about students' attentional processing and performance when solving arithmetic expressions (as interpreted above), there is the possibility that our study design also contributed to these findings. Because our undergraduate sample was taken from a STEM‐focused university with a strong emphasis on mathematics, our participants likely found the current study's arithmetic expressions to be relatively easy to solve; thus, the lack of differences in percent of correct responses could be the result of a ceiling effect. Future studies may want to consider utilizing more complex arithmetic expressions that would be more challenging for participants with higher arithmetic knowledge to compare with simpler arithmetic expressions.</p> <hd id="AN0179320539-35">Implications for Learning</hd> <p>Order of operations is a foundational skill that is practiced throughout formal schooling and a precursor to success in higher‐level mathematics. This skill asks students to identify and use deep mathematical structures that are relevant to solving the problem (Kieran, [<reflink idref="bib12" id="ref65">12</reflink>]; National Governors Association Center for Best Practices, [<reflink idref="bib19" id="ref66">19</reflink>]; Venkat, Askew, Watson, & Mason, [<reflink idref="bib29" id="ref67">29</reflink>]); however, many students are unable to do so, and may instead fall back on applying calculational heuristics in inappropriate situations, which can lead to inaccurate or inefficient solutions. If visual perceptual cues can direct attention to important mathematical structures as suggested by the interaction between HOO position and superfluous brackets, then perceptual cues may be a useful tool for helping students learn important concepts of order of operations. Educators can utilize superfluous brackets and similar cues to guide students' attention to relevant features and facilitate efficient solution strategies.</p> <p>The difference in findings between the current study's undergraduate sample and our prior study's middle school sample (Ngo et al., [<reflink idref="bib20" id="ref68">20</reflink>]) suggests that superfluous brackets could be a remedial tool for learners who are acquiring knowledge of order‐of‐operations. Still, the impact of superfluous cues may be limited: we only found a moderating effect when the HOO was in the center, not the right. Thus, the left‐to‐right tendency may be particularly strong as the HOO moves farther from the left, meaning that superfluous brackets may need to be combined with other instructional scaffolds to support mathematical performance. Superfluous brackets may also be a less effective perceptual cue for undergraduates compared to younger students, as undergraduates' mathematical knowledge and heuristic biases may be more ingrained.</p> <p>Assuming that eye fixations reflect students' attentional processes, similar methods that track students' attentional focuses could eventually be scaled into classroom settings to examine whether students who are learning arithmetic use the same solution approaches as observed here in expert adults. Being able to detect how novices' attentional patterns differ from experts, as well as how they change with expertise, may provide implicit instructional markers (detected through eye tracking software or self‐reports, as used in Green, Lemaire, & Dufau, [<reflink idref="bib5" id="ref69">5</reflink>] or Verschaffel, Corte, Gielen, & Struyf, [<reflink idref="bib30" id="ref70">30</reflink>]) that educators can use to better understand students' learning progressions.</p> <hd id="AN0179320539-36">CONCLUSION</hd> <p>The current study's findings demonstrate that visual information, such as the presence of superfluous brackets, can act as perceptual cues that influence how students process arithmetic expressions. Specifically, these perceptual cues can communicate information about calculational order above what is communicated by order of operations rules such as higher‐order operators. Using eye tracking methodology, we provide early insights into how superfluous brackets can moderate the effect of HOO positions, impacting the underlying cognitive processes as adults solve arithmetic expressions, as well as their subsequent mathematical performance.</p> <hd id="AN0179320539-37">Funding Information</hd> <p>No funding was received from any organization for the submitted work.</p> <hd id="AN0179320539-38">Conflict of Interest Statement</hd> <p>The authors have stated explicitly that there are no conflicts of interest in connection with this paper.</p> <hd id="AN0179320539-39">Ethics Approval</hd> <p>The study received approval from Worcester Polytechnic Institute's Institutional Review Board.</p> <hd id="AN0179320539-40">Data Availability Statement</hd> <p>The study hypotheses, design, and data analysis plan were preregistered on Open Science Framework (OSF) at https://osf.io/fbwxy/?view%5fonly=2eb85206fc604680afc18adeea9d9440. All data, analysis code, and research materials are available on the OSF project page.</p> <hd id="AN0179320539-41">A APPENDIX FULL PROBLEM LIST WITH TRAINING ITEMS IN PRE‐RANDOMIZED ORDER</hd> <p></p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Condition</th><th align="center">Problem</th><th align="center">Answer</th></tr></thead><tbody valign="top"><tr><td align="left">Training 1</td><td align="char" char=".">7 + 2 + 5*3</td><td align="char" char=".">24</td></tr><tr><td align="left">Training 2</td><td align="char" char=".">6/3 + 2–1</td><td align="char" char=".">3</td></tr><tr><td align="left">Training 3</td><td align="char" char=".">7 + 8*4–2</td><td align="char" char=".">37</td></tr><tr><td align="left">Brackets‐Center</td><td align="char" char=".">6 + (4/4) + 1</td><td align="char" char=".">8</td></tr><tr><td align="left">Brackets‐Right</td><td align="char" char=".">4 + 1 + (8/4)</td><td align="char" char=".">7</td></tr><tr><td align="left">No Brackets‐Right</td><td align="char" char=".">7–5 + 5*2</td><td align="char" char=".">12</td></tr><tr><td align="left">No Brackets‐Center</td><td align="char" char=".">5 + 2*2 + 3</td><td align="char" char=".">12</td></tr><tr><td align="left">Brackets‐Left</td><td align="char" char=".">(2*3) + 4 + 3</td><td align="char" char=".">13</td></tr><tr><td align="left">Brackets‐Center</td><td align="char" char=".">8 + (5/1)‐4</td><td align="char" char=".">9</td></tr><tr><td align="left">No Brackets‐Left</td><td align="char" char=".">5*2 + 7–6</td><td align="char" char=".">11</td></tr><tr><td align="left">No Brackets‐Center</td><td align="char" char=".">8 + 3/3–4</td><td align="char" char=".">5</td></tr><tr><td align="left">No Brackets‐Center</td><td align="char" char=".">6 + 8/2 + 1</td><td align="char" char=".">11</td></tr><tr><td align="left">Brackets‐Left</td><td align="char" char=".">(8/4) + 5 + 1</td><td align="char" char=".">8</td></tr><tr><td align="left">Brackets‐Right</td><td align="char" char=".">5–6 + (5*2)</td><td align="char" char=".">9</td></tr><tr><td align="left">Brackets‐Center</td><td align="char" char=".">7 + (3*2)‐6</td><td align="char" char=".">7</td></tr><tr><td align="left">Brackets‐Right</td><td align="char" char=".">6–4 + (3/1)</td><td align="char" char=".">5</td></tr><tr><td align="left">No Brackets‐Left</td><td align="char" char=".">8/4 + 6 + 1</td><td align="char" char=".">9</td></tr><tr><td align="left">No Brackets‐Right</td><td align="char" char=".">5 + 2 + 2*3</td><td align="char" char=".">13</td></tr><tr><td align="left">No Brackets‐Center</td><td align="char" char=".">7 + 5*1–6</td><td align="char" char=".">6</td></tr><tr><td align="left">No Brackets‐Left</td><td align="char" char=".">2*3 + 5 + 3</td><td align="char" char=".">14</td></tr><tr><td align="left">No Brackets‐Left</td><td align="char" char=".">3/1 + 8–4</td><td align="char" char=".">7</td></tr><tr><td align="left">No Brackets‐Right</td><td align="char" char=".">8–5 + 3/1</td><td align="char" char=".">6</td></tr><tr><td align="left">Brackets‐Center</td><td align="char" char=".">5 + (1*3) + 3</td><td align="char" char=".">11</td></tr><tr><td align="left">Brackets‐Right</td><td align="char" char=".">3 + 3 + (2*3)</td><td align="char" char=".">12</td></tr><tr><td align="left">Brackets‐Left</td><td align="char" char=".">(3/1) + 7–4</td><td align="char" char=".">6</td></tr><tr><td align="left">Brackets‐Left</td><td align="char" char=".">(5*2) + 6–6</td><td align="char" char=".">10</td></tr><tr><td align="left">No Brackets‐Right</td><td align="char" char=".">6 + 2 + 8/4</td><td align="char" char=".">10</td></tr></tbody></table> </ephtml> </p> <ref id="AN0179320539-42"> <title> REFERENCES </title> <blist> <bibl id="bib1" idref="ref20" type="bt">1</bibl> <bibtext> Ayres, P. (2000). An analysis of bracket expansion errors. In T. Nakahara, & M. Koyama (Eds.), Proceedings of the 24th conference of the International Group for the Psychology of mathematics education. (Vol. 2, pp. 25 – 31). Hiroshima, Japan : PME.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref30" type="bt">2</bibl> <bibtext> Banerjee, R., & Subramaniam, K. (2005). Developing procedure and structure sense of arithmetic expressions. In H. L. Chick, & J. L. Vincent (Eds.), Proceedings of the 29th conference of the International Group for the Psychology of mathematics education. (Vol. 2, pp. 121 – 128). Melbourne, Australia: PME.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref33" type="bt">3</bibl> <bibtext> Blando, J. A., Kelly, A. E., Schneider, B. R., & Sleeman, D. (1989). Analyzing and modeling arithmetic errors. Journal for Research in Mathematics Education, 20 (3), 301 – 308. https://doi.org/10.2307/749518</bibtext> </blist> <blist> <bibl id="bib4" idref="ref6" type="bt">4</bibl> <bibtext> Goldstone, R. L., Marghetis, T., Weitnauer, E., Ottmar, E. R., & Landy, D. (2017). Adapting perception, action, and technology for mathematical reasoning. Current Directions in Psychological Science, 26 (5), 434 – 441. https://doi.org/10.1177/0963721417704888</bibtext> </blist> <blist> <bibl id="bib5" idref="ref69" type="bt">5</bibl> <bibtext> Green, H. J., Lemaire, P., & Dufau, S. (2007). Eye movement correlates of younger and older adults' strategies for complex addition. Acta Psychologica, 125 (3), 257 – 278. https://doi.org/10.1016/j.actpsy.2006.08.001</bibtext> </blist> <blist> <bibl id="bib6" idref="ref19" type="bt">6</bibl> <bibtext> Gunnarsson, R., Sönnerhed, W. W., & Hernell, B. (2016). Does it help to use mathematically superfluous brackets when teaching the rules for the order of operations? Educational Studies in Mathematics, 92 (1), 91 – 105. https://doi.org/10.1007/s10649‐015‐9667‐2</bibtext> </blist> <blist> <bibl id="bib7" idref="ref1" type="bt">7</bibl> <bibtext> Harrison, A., Smith, H., Hulse, T., & Ottmar, E. (2020). Spacing out!: Manipulating spatial features in mathematical expressions affects performance. Journal of Numerical Cognition, 6 (2), 186 – 203. https://doi.org/10.5964/jnc.v6i2.243</bibtext> </blist> <blist> <bibl id="bib8" idref="ref21" type="bt">8</bibl> <bibtext> Hewitt, D. (2005). Chinese whispers ‐ algebra style: Grammatical, notational, mathematical and activity tensions. In H. L. Chick, & J. L. Vincent (Eds.), Proceedings of the 29th conference of the International Group for the Psychology of mathematics education. (Vol. 3, pp. 129 – 136). Melbourne, Australia: PME.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref8" type="bt">9</bibl> <bibtext> Hoch, M., & Dreyfus, T. (2004). Structure sense in high school algebra: The effect of brackets. In M. J. Hoines, & A. B. Fuglestad (Eds.), Proceedings of the 28th conference of the International Group for the Psychology of mathematics education. (Vol. 3, pp. 49 – 56). Bergen, Norway: PME.</bibtext> </blist> <blist> <bibtext> Kellman, P. J., & Massey, C. M. (2013). Perceptual learning, cognition, and expertise. In B. H. Ross (Ed.), The psychology of learning and motivation. (pp. 117 – 165). Waltham, MA: Elsevier Academic Press.</bibtext> </blist> <blist> <bibtext> Kieran, C. (1979). Children's operational thinking within the context of bracketing and the order of operations. In D. Tall (Ed.), Proceedings of the 3rd conference of the International Group for the Psychology of mathematics education. (pp. 128 – 133). Coventry, UK: PME.</bibtext> </blist> <blist> <bibtext> Kieran, C. (1989). The early learning of algebra: A structural perspective. In S. Wagner, & C. Kieran (Eds.), Research issues in the learning and teaching of algebra: The research agenda for mathematics education. (Vol. 4, pp. 33 – 56). New York, NY: NCTM and Lawrence Erlbaum Associates. https://doi.org/10.4324/9781315044378‐4</bibtext> </blist> <blist> <bibtext> Kirshner, D., & Awtry, T. (2004). Visual salience of algebraic transformations. Journal for Research in Mathematics Education, 35 (4), 224 – 257. https://doi.org/10.2307/30034809</bibtext> </blist> <blist> <bibtext> Landy, D., & Goldstone, R. L. (2007). How abstract is symbolic thought? Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 720 – 733. https://doi.org/10.1037/0278‐7393.33.4.720</bibtext> </blist> <blist> <bibtext> Landy, D., Jones, M. N., & Goldstone, R. L. (2008). How the appearance of an operator affects its formal precedence. In B. C. Love, K. McRae & V. M. Sloutsky (Eds.), Proceedings of the 30th annual conference of the cognitive science society. (pp. 2110 – 2115). Washington, DC: Cognitive Science Society.</bibtext> </blist> <blist> <bibtext> Linchevski, L., & Livneh, D. (1999). Structure sense: The relationship between algebraic and numerical contexts. Educational Studies in Mathematics, 40 (2), 173 – 196. https://doi.org/10.1023/A:1003606308064</bibtext> </blist> <blist> <bibtext> Marchini, C., & Papadopoulos, I. (2011). Are useless brackets useful tools for teaching? In B. Ubuz (Ed.), Proceedings of the 35th conference of the International Group for the Psychology of mathematics education. (Vol. 3, pp. 185 – 192). Ankara, Turkey: PME.</bibtext> </blist> <blist> <bibtext> Marghetis, T., Landy, D., & Goldstone, R. L. (2016). Mastering algebra retrains the visual system to perceive hierarchical structure in equations. Cognitive Research: Principles and Implications, 1 (1), 25. https://doi.org/10.1186/s41235‐016‐0020‐9</bibtext> </blist> <blist> <bibtext> National Governors Association Center for Best Practices, Council of Chief State School Officers (2010). Common Core state standards. Washington, DC: Authors.</bibtext> </blist> <blist> <bibtext> Ngo, V., Lacera, L. P., Closser, A. H., & Ottmar, E. (2023). The effects of operator position and superfluous brackets on student performance in simple arithmetic. Journal of Numerical Cognition, 9 (1), 107 – 128. https://doi.org/10.5964/jnc.9535</bibtext> </blist> <blist> <bibtext> Olsen, A. (2012) The Tobii I‐VT Fixation Filter: Algorithm Description. Stockholm, Sweden: Tobii Technology. https://<ulink href="http://www.vinis.co.kr/ivt%5ffilter.pdf">www.vinis.co.kr/ivt%5ffilter.pdf</ulink></bibtext> </blist> <blist> <bibtext> Papadopoulos, I., & Gunnarsson, R. (2018). The use of 'mental' brackets when calculating arithmetic expressions. In E. Bergqvist, M. Österholm, C. Granberg & L. Sumpter (Eds.), Proceedings of the 42nd conference of the International Group for the Psychology of mathematics education. (Vol. 3, pp. 451 – 458). Umea, Sweden: PME.</bibtext> </blist> <blist> <bibtext> Papadopoulos, I., & Gunnarsson, R. (2020). Exploring the way rational expressions trigger the use of "mental" brackets by primary school students. Educational Studies in Mathematics, 103 (2), 191 – 207. https://doi.org/10.1007/s10649‐019‐09929‐z</bibtext> </blist> <blist> <bibtext> Papadopoulos, I., & Thoma, A. (2022). Mental brackets and their use by high school students in arithmetic and algebra. International Journal of Science and Mathematics Education, 21, 1197–1218. https://doi.org/10.1007/s10763‐022‐10298‐y</bibtext> </blist> <blist> <bibtext> Schindler, M., & Lilienthal, A. J. (2019). Domain‐specific interpretation of eye tracking data: Towards a refined use of the eye‐mind hypothesis for the field of geometry. Educational Studies in Mathematics, 101, 123 – 139. https://doi.org/10.1007/s10649‐019‐9878‐z</bibtext> </blist> <blist> <bibtext> Schneider, E., Muruyama, M., Dehaene, S., & Sigman, M. (2012). Eye gaze reveals a fast, parallel extraction of the syntax of arithmetic formulas. Cognition, 125, 475 – 490. https://doi.org/10.1016/j.cognition.2012.06.015</bibtext> </blist> <blist> <bibtext> Strohmaier, A. R., MacKay, K. J., Obersteiner, A., & Reiss, K. M. (2020). Eye‐tracking methodology in mathematics education research: A systematic literature review. Educational Studies in Mathematics, 104 (2), 147 – 200. https://doi.org/10.1007/s10649‐020‐09948‐1</bibtext> </blist> <blist> <bibtext> Tobii Pro. (n.d.). Tobii Pro Lab. https://<ulink href="http://www.tobiipro.com/product‐listing/tobii‐pro‐lab/">www.tobiipro.com/product‐listing/tobii‐pro‐lab/</ulink></bibtext> </blist> <blist> <bibtext> Venkat, H., Askew, M., Watson, A., & Mason, J. (2019). Architecture of mathematical structure. For the Learning of Mathematics, 39 (1), 13 – 17. https://flm‐journal.org/Articles/15CDA7D47D380050F5587BF8C2BCB8.pdf</bibtext> </blist> <blist> <bibtext> Verschaffel, L., Corte, E. d., Gielen, E., & Struyf, E. (1994). Clever rearrangement strategies in children's mental arithmetic: A confrontation of eye‐movement data and verbal protocols. In J. E. H. Van Luit (Ed.), Research on learning and instruction of mathematics in kindergarten and primary school. (pp. 153 – 180). Doetinchem, The Netherlands: Graviant.</bibtext> </blist> </ref> <aug> <p>By Alena Egorova; Vy Ngo; Allison S. Liu; Molly Mahoney; Justine Moy and Erin Ottmar</p> <p>Reported by Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib10" firstref="ref2"></nolink> <nolink nlid="nl2" bibid="bib13" firstref="ref3"></nolink> <nolink nlid="nl3" bibid="bib14" firstref="ref4"></nolink> <nolink nlid="nl4" bibid="bib18" firstref="ref5"></nolink> <nolink nlid="nl5" bibid="bib17" firstref="ref9"></nolink> <nolink nlid="nl6" bibid="bib20" firstref="ref10"></nolink> <nolink nlid="nl7" bibid="bib23" firstref="ref11"></nolink> <nolink nlid="nl8" bibid="bib24" firstref="ref13"></nolink> <nolink nlid="nl9" bibid="bib16" firstref="ref16"></nolink> <nolink nlid="nl10" bibid="bib11" firstref="ref22"></nolink> <nolink nlid="nl11" bibid="bib27" firstref="ref23"></nolink> <nolink nlid="nl12" bibid="bib25" firstref="ref24"></nolink> <nolink nlid="nl13" bibid="bib15" firstref="ref25"></nolink> <nolink nlid="nl14" bibid="bib26" firstref="ref26"></nolink> <nolink nlid="nl15" bibid="bib28" firstref="ref43"></nolink> <nolink nlid="nl16" bibid="bib19" firstref="ref44"></nolink> <nolink nlid="nl17" bibid="bib21" firstref="ref46"></nolink> <nolink nlid="nl18" bibid="bib22" firstref="ref58"></nolink> <nolink nlid="nl19" bibid="bib12" firstref="ref65"></nolink> <nolink nlid="nl20" bibid="bib29" firstref="ref67"></nolink> <nolink nlid="nl21" bibid="bib30" firstref="ref70"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1437318
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Mathematics Presentation Matters: How Superfluous Brackets and Higher-Order Operator Position in Mathematics Can Impact Arithmetic Performance
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Alena+Egorova%22">Alena Egorova</searchLink><br /><searchLink fieldCode="AR" term="%22Vy+Ngo%22">Vy Ngo</searchLink><br /><searchLink fieldCode="AR" term="%22Allison+S%2E+Liu%22">Allison S. Liu</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1075-2575">0000-0003-1075-2575</externalLink>)<br /><searchLink fieldCode="AR" term="%22Molly+Mahoney%22">Molly Mahoney</searchLink><br /><searchLink fieldCode="AR" term="%22Justine+Moy%22">Justine Moy</searchLink><br /><searchLink fieldCode="AR" term="%22Erin+Ottmar%22">Erin Ottmar</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-9487-7967">0000-0002-9487-7967</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Mind%2C+Brain%2C+and+Education%22"><i>Mind, Brain, and Education</i></searchLink>. 2024 18(3):258-269.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 12
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22STEM+Education%22">STEM Education</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematical+Formulas%22">Mathematical Formulas</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Instruction%22">Mathematics Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Eye+Movements%22">Eye Movements</searchLink><br /><searchLink fieldCode="DE" term="%22Arithmetic%22">Arithmetic</searchLink><br /><searchLink fieldCode="DE" term="%22Computation%22">Computation</searchLink><br /><searchLink fieldCode="DE" term="%22Thinking+Skills%22">Thinking Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Symbols+%28Mathematics%29%22">Symbols (Mathematics)</searchLink><br /><searchLink fieldCode="DE" term="%22Interference+%28Learning%29%22">Interference (Learning)</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Achievement%22">Mathematics Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Processes%22">Learning Processes</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/mbe.12421
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1751-2271<br />1751-228X
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Perceptual learning theory suggests that perceptual grouping in mathematical expressions can direct students' attention toward specific parts of problems, thus impacting their mathematical reasoning. Using in-lab eye tracking and a sample of 85 undergraduates from a STEM-focused university, we investigated how higher-order operator position (HOO; i.e., multiplication/division operators and the presence of superfluous brackets impacted students' time to first fixation to the HOO, response time, and percent of correct responses). Students solved order-of-operations problems presented in six ways (3 HOO positions × presence of brackets). We found that HOO position and presence of superfluous brackets had separate and combined impacts on calculating arithmetic expressions. Superfluous brackets most influenced undergraduates' performance when higher-order operators were located in the center of mathematical expressions. Implications for learning and future directions are discussed about observing eye movements and gaining insights into students' processes when solving arithmetic expressions.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Note
  Label: Notes
  Group: Note
  Data: https://osf.io/fbwxy/?view_only=2eb85206fc604680afc18adeea9d9440
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1437318
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1437318
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/mbe.12421
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
        StartPage: 258
    Subjects:
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: STEM Education
        Type: general
      – SubjectFull: Mathematical Formulas
        Type: general
      – SubjectFull: Mathematics Instruction
        Type: general
      – SubjectFull: Eye Movements
        Type: general
      – SubjectFull: Arithmetic
        Type: general
      – SubjectFull: Computation
        Type: general
      – SubjectFull: Thinking Skills
        Type: general
      – SubjectFull: Symbols (Mathematics)
        Type: general
      – SubjectFull: Interference (Learning)
        Type: general
      – SubjectFull: Mathematics Achievement
        Type: general
      – SubjectFull: Learning Processes
        Type: general
    Titles:
      – TitleFull: Mathematics Presentation Matters: How Superfluous Brackets and Higher-Order Operator Position in Mathematics Can Impact Arithmetic Performance
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Alena Egorova
      – PersonEntity:
          Name:
            NameFull: Vy Ngo
      – PersonEntity:
          Name:
            NameFull: Allison S. Liu
      – PersonEntity:
          Name:
            NameFull: Molly Mahoney
      – PersonEntity:
          Name:
            NameFull: Justine Moy
      – PersonEntity:
          Name:
            NameFull: Erin Ottmar
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 1751-2271
            – Type: issn-electronic
              Value: 1751-228X
          Numbering:
            – Type: volume
              Value: 18
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Mind, Brain, and Education
              Type: main
ResultId 1