View in EDS HTML Full Text PDF Full Text

A Meta-Analysis of STEM Integration on Student Academic Achievement

Saved in:
Bibliographic Details
Title: A Meta-Analysis of STEM Integration on Student Academic Achievement
Language: English
Authors: Shuqi Zhou, Zehua Dong, Hui Hui Wang, Ming Ming Chiu
Source: Research in Science Education. 2025 55(5):1273-1302.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 30
Publication Date: 2025
Document Type: Journal Articles
Information Analyses
Descriptors: Meta Analysis, STEM Education, Academic Achievement, Effect Size, Problem Based Learning, Science Achievement, Mathematics Achievement, Engineering Education, Collectivism, Individualism, Instructional Design, Integrated Curriculum
DOI: 10.1007/s11165-024-10216-y
ISSN: 0157-244X
1573-1898
Abstract: This meta-analysis examined whether learning outcomes differ (a) for STEM integration versus traditional instruction and (b) across STEM integration implementations. Based on 79 effect sizes from 40 studies of 15,577 students, those learning via STEM integration outperformed other students on academic achievement tests (g = 0.661; 95% CI [0.548, 0.774]). The effect sizes of STEM integration on achievement were largest for context integration, smaller for content integration, and smallest for tool integration. They were largest for inquiry-based learning, and progressively smaller for problem-based learning, designed-based learning, and project-based learning. They were largest for STEM subject achievement, and progressively smaller for science achievement, math achievement, and engineering achievement. They were larger for collectivist countries than for individualistic countries. Engineering design skills and grade level were not significant moderators. These results can inform integrated STEM instructional design and improve student learning.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1489580
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGqD4m2EDqhml2740_zhlEsAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDEYZ1Bk6j7msn9ZpvQIBEICBmsDKZnMvrLNBi4uXs92FF3olLwh-tItMirL8OwL7iXRE1FjKU2tpp2J1sN9aPtfOMp5F68pEbUwy6_LoK1_1VbyrKkXL2upPeUkhR0Yv-OQcuBU6f-4Siu0LBJFup6V5DcTU6LsrvlMDL-uc3J6uLmMMOBvLkN7rnrfhqdD9RZp40w1RVs3Td5kUWJxYp-FlEQlp3L0I5nbXKHA=
Text:
  Availability: 1
  Value: <anid>AN0188452166;g7201oct.25;2025Oct07.03:58;v2.2.500</anid> <title id="AN0188452166-1">A Meta-analysis of STEM Integration on Student Academic Achievement </title> <p>This meta-analysis examined whether learning outcomes differ (a) for STEM integration versus traditional instruction and (b) across STEM integration implementations. Based on 79 effect sizes from 40 studies of 15,577 students, those learning via STEM integration outperformed other students on academic achievement tests (g = 0.661; 95% CI [0.548, 0.774]). The effect sizes of STEM integration on achievement were largest for context integration, smaller for content integration, and smallest for tool integration. They were largest for inquiry-based learning, and progressively smaller for problem-based learning, designed-based learning, and project-based learning. They were largest for STEM subject achievement, and progressively smaller for science achievement, math achievement, and engineering achievement. They were larger for collectivist countries than for individualistic countries. Engineering design skills and grade level were not significant moderators. These results can inform integrated STEM instructional design and improve student learning.</p> <p>Keywords: STEM integration; Achievement; Meta-analysis; Education Curriculum and Pedagogy Specialist Studies In Education</p> <p>Copyright comment Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.</p> <p>Integrated Science, Technology, Engineering, and Mathematics (STEM) education "combine[s] the four disciplines of science, technology, engineering, and mathematics into one class, unit, or lesson that is based on connections among these disciplines and real-world problems" (Moore & Smith, [<reflink idref="bib67" id="ref1">67</reflink>], p.5). Past studies have shown that STEM integration can increase students' STEM interests (e.g., Struyf et al., [<reflink idref="bib99" id="ref2">99</reflink>]), promote positive attitudes toward STEM (Guzey et al., [<reflink idref="bib28" id="ref3">28</reflink>]), improve conceptual understanding (e.g., Aminger et al., [<reflink idref="bib6" id="ref4">6</reflink>]), foster higher-order thinking skills (English et al., [<reflink idref="bib23" id="ref5">23</reflink>]), raise learning outcomes (e.g., Robinson et al., [<reflink idref="bib84" id="ref6">84</reflink>]), and aid college preparation and STEM-related career choices (NRC, [<reflink idref="bib72" id="ref7">72</reflink>]). Although STEM integration improved 4th grade students' science and mathematics achievements (Acar et al., [<reflink idref="bib1" id="ref8">1</reflink>]), it did not affect high school students' technology, mathematics, and science achievements (Merrill, [<reflink idref="bib64" id="ref9">64</reflink>])—even worse, it showed a negative effect on high school students' science achievements (McGehee, [<reflink idref="bib62" id="ref10">62</reflink>]). Different types of integrated STEM instruction might account for such results: (a) types of integration (context, content, tool; Moore et al., [<reflink idref="bib66" id="ref11">66</reflink>]), (b) learning activities (inquiry-based, problem-based, design-based, project-based; Roehrig, et al., [<reflink idref="bib85" id="ref12">85</reflink>]), (c) engineering design (Bryan et al., [<reflink idref="bib15" id="ref13">15</reflink>]), and so on.</p> <p>Moreover, past meta-analyses omitted many studies or had technical flaws. Following Becker and Park's ([<reflink idref="bib11" id="ref14">11</reflink>]) systematic review of STEM integration, meta-analyses of subsets of STEM activities on learning outcomes excluded some grade levels, academic subjects, or regions (early childhood education, ES = 0.556, Yücelyiğit & Toker, [<reflink idref="bib118" id="ref15">118</reflink>]; mathematics achievement, Hedge's <emph>g</emph> = 0.242, Siregar et al., [<reflink idref="bib96" id="ref16">96</reflink>]; Asian students, Cohen's <emph>d</emph> = 0.69, Wahono et al., [<reflink idref="bib109" id="ref17">109</reflink>]). Moreover, three meta-analyses had technical flaws. Saraç ([<reflink idref="bib90" id="ref18">90</reflink>]) did not specify clear selection criteria. Furthermore, Zeng and Yao ([<reflink idref="bib119" id="ref19">119</reflink>]) did a limited search for original studies, and did not consider pre-post effect sizes. Siregar et al. ([<reflink idref="bib96" id="ref20">96</reflink>]) did not consistently use fixed effects and mixed effects for moderators.</p> <p>Thus, this up-to-date meta-analysis uses all available original studies of STEM integration and K-12 student achievement published by January 2024 to determine the effects of STEM integration on student achievement. To account for past studies' different results, we test for moderator effects. These results will inform STEM education researchers and teachers to make wiser choices as they implement STEM integration.</p> <hd id="AN0188452166-2">Theoretical Perspective</hd> <p>Bybee ([<reflink idref="bib16" id="ref21">16</reflink>]) broadly defines STEM education to include any event, policy, program, or practices with one or more of the STEM disciplines, but we prefer Vasquez et al.'s ([<reflink idref="bib108" id="ref22">108</reflink>]) disciplinary knowledge continuum of a single discipline, multidisciplinary, interdisciplinary, and transdisciplinary. Most classes stick to a single subject like math. By contrast, students tackle separate subjects before combining them to solve a problem in <emph>multidisciplinary</emph> approaches. <emph>Interdisciplinary</emph> approaches goes a step further, blending concepts across fields. Lastly, students combine disciplines to address real-world problems in <emph>transdisciplinary</emph> approaches. In this paper, we embrace a transdisciplinary definition of STEM integration as mixing two or more STEM disciplines to address real-world problems. For this study, <emph>learning outcome</emph> (aka <emph>achievement</emph>) captures how well a student has accomplished specific learning goals via STEM integration.</p> <hd id="AN0188452166-3">STEM Integration and Learning</hd> <p>STEM integration grabs students' interest, which motivates them to sharpen their higher-order thinking skills and learn more. Each subject has its fans, but when we mix them, more students are interested in at least one of them. Some are hooked by the real-world STEM problem or its relevance, even if the subjects do not grab them. Solving a real problem can be satisfying and have real-world benefits, which can motivate still more students. So, compared to a regular school task in one subject, a STEM project gets more students excited and eager to learn (Struyf et al., [<reflink idref="bib99" id="ref23">99</reflink>]). Also, STEM projects offer more to learn than single-subject tasks. Moreover, real-world problems are messier and more challenging than school exercises, offering richer learning experiences (though they can be overwhelming). As these projects engage more students and provide more learning opportunities, students can learn more from them than from traditional schoolwork (Robinson et al., [<reflink idref="bib84" id="ref24">84</reflink>]). Hence, we propose hypothesis H-1.</p> <p></p> <ulist> <item> H-1. Students in STEM integration activities outperform other students on learning outcomes.</item> </ulist> <hd id="AN0188452166-4">Moderators</hd> <p>Many scholars are studying STEM education, but they do not agree on a single framework or clear definition of STEM integration (Moore et al., [<reflink idref="bib66" id="ref25">66</reflink>]). Some researchers detail their integrated STEM learning units but not the instructional principles behind their design (e.g., Barrett et al., [<reflink idref="bib10" id="ref26">10</reflink>]; Gentile et al., [<reflink idref="bib27" id="ref27">27</reflink>]). Conversely, others give detailed accounts of their teaching methods but lack a theoretical basis (e.g., Moore & Smith, [<reflink idref="bib67" id="ref28">67</reflink>]). Hence, Moore et al.'s ([<reflink idref="bib66" id="ref29">66</reflink>]) reviewed the literature to see how researchers define and use STEM integration. They synthesized 109 articles, noting that most agree on the importance of tackling real-world problems. However, views differ on: (a) the degree of integration, (b) learning activities, (c) the number of included disciplines, (d) implementation strategies, and (e) the roles of individual disciplines (Moore et al., [<reflink idref="bib66" id="ref30">66</reflink>]). The original studies had enough data to test the degree of integration and learning activities for moderation effects (but lacked information on the number of disciplines, roles of each discipline, and implementation strategies). Thus, we test whether these moderators and others (demographics, study design) account for the different effect sizes in past studies.</p> <hd id="AN0188452166-5">Degree of Integration: Context, Content, and Tool Integration</hd> <p>STEM integration can occur across contexts, content or tools. <emph>Context integration</emph> uses a meaningful, relevant, concrete, authentic problem situation in one subject to learn about another (Bryan et al., [<reflink idref="bib15" id="ref31">15</reflink>]; Moore & Smith, [<reflink idref="bib67" id="ref32">67</reflink>]). For example, a teacher (Ms. T) teaches her students about temperature (physics). She asks them to find the temperature ranges at which poodles thrive (biology). As Ms. T brings real life into the classroom, learning about temperature becomes meaningful to her students, so they are more eager to learn (Nadelson & Seifert, [<reflink idref="bib68" id="ref33">68</reflink>]). Furthermore, they create more neural connections about temperature, cats and dogs in their brains, which aids their later recall and novel uses (Li & Wang, [<reflink idref="bib54" id="ref34">54</reflink>]). By using these ideas across diverse problems, they learn how to apply them in new ways (<emph>transfer</emph>, Perkins & Salomon, [<reflink idref="bib79" id="ref35">79</reflink>]).</p> <p>Whereas context integration only uses a context from another subject, <emph>content integration</emph> mixes ideas across subjects (Wang & Knobloch, [<reflink idref="bib110" id="ref36">110</reflink>], [<reflink idref="bib111" id="ref37">111</reflink>]). For example, Ms. T asks her students to model heat indices (H) of temperature (T) and relative humidity (R) across environments that poodles can tolerate. To understand their relations, students must integrate physics and math (H = f[T, R]), which helps them see how these subjects fit together. Hence, they also build more neural connections across subjects as well as with the real-world situation (Li & Wang, [<reflink idref="bib54" id="ref38">54</reflink>]). All of this can help them solve complex problems (Siverling et al., [<reflink idref="bib95" id="ref39">95</reflink>]). (Roehrig et al. [[<reflink idref="bib85" id="ref40">85</reflink>]] argued for intertwining of content and context integration to make STEM integration more meaningful to students.)</p> <p>During <emph>tool integration</emph>, teachers help students learn one subject with a novel tool from another one (without necessarily adding its context or content; Moore et al., [<reflink idref="bib66" id="ref41">66</reflink>]). For example, Ms. T asks her students to use <emph>thermocouples</emph>(Bajzek, [<reflink idref="bib9" id="ref42">9</reflink>]) to measure temperatures in different places—kitchen, pool, beach—and think about where their poodles can live. Thus, students learn how to use tools to solve real-world problems. Learning how to use the thermocouple to measure the temperature was the focus of the instruction. Although chemistry knowledge (esp. relation between thermocouples's two metals) spurred the invention of the thermometer, Ms. T does not explain its origin, so her students do not learn the links between mercury and temperature.</p> <p>No past study showed whether the degree of STEM integration affects student learning. As context and content integration offer more links across subjects than tool integration, student can learn more from the first two. Moreover, teachers might plan for and use context integration more easily than content integration. As many teachers did not study a STEM subject in college, they do not know enough STEM content or feel confident enough (<emph>self-efficacy</emph>) to guide students in project-based learning or design-based learning (Brophy et al., [<reflink idref="bib14" id="ref43">14</reflink>]; Frank et al., [<reflink idref="bib25" id="ref44">25</reflink>]; Nadelson et al., [<reflink idref="bib69" id="ref45">69</reflink>]). For example, some teachers do not know enough about engineering to include it in their lessons (Hamad et al., [<reflink idref="bib32" id="ref46">32</reflink>]). Time and curriculum limits (Helle et al., [<reflink idref="bib33" id="ref47">33</reflink>]) also obstruct content integration (e.g., integrating math into science/engineering classes; Roehrig et al., [<reflink idref="bib85" id="ref48">85</reflink>]).</p> <p>In short, (a) context integration improves learning outcomes and is easy to use, (b) content integration is harder to use but still aids learning, and (c) tool integration offers the least benefit. While context integration, content integration, and tool integration can overlap, they differ substantially. So, we test whether context, content, or tools integration accounts for the differences in results across studies. Thus, we propose this hypothesis.</p> <p></p> <ulist> <item> H-2. Student learning outcomes are highest for context integration, lower for content integration, and lowest for tool integration.</item> </ulist> <hd id="AN0188452166-6">Learning Activities</hd> <p>Some teachers use student-centered design (i.e., problem-based learning, project-based learning, design-based learning) to help students learn. They ask them to solve complex, real-world problems with scientific methods or engineering design processes (Bryan et al., [<reflink idref="bib15" id="ref49">15</reflink>]; Wang & Knobloch, [<reflink idref="bib111" id="ref50">111</reflink>]). In these lessons, students learn and apply cross-cutting concepts, think critically, and solve problems across contexts (e.g., Margot & Kettler, [<reflink idref="bib58" id="ref51">58</reflink>]; Robinson et al., [<reflink idref="bib84" id="ref52">84</reflink>]). They connect concepts and practices across subjects through a common problem (Roehrig, et al., [<reflink idref="bib85" id="ref53">85</reflink>]; Vasquez et al., [<reflink idref="bib107" id="ref54">107</reflink>]) to learn more efficiently (e.g., You, [<reflink idref="bib117" id="ref55">117</reflink>]). Teachers have taught STEM integration via problem-based, inquiry-based, project-based or design-based learning (Moore, et al., [<reflink idref="bib66" id="ref56">66</reflink>]). In all of these methods, students build understanding and meaning from prior experiences, supported by tools in their teachers' learning environment (<emph>constructivism;</emph> Cobern, [<reflink idref="bib19" id="ref57">19</reflink>]).</p> <p>Students can learn by working on a complex problem (<emph>problem-based learning</emph>; Servant-Miklos, [<reflink idref="bib93" id="ref58">93</reflink>]). For example, hot, humid days in Phoenix, Arizona can harm poodles (and humans!). So, students tried to identify such days. They measured the temperature, humidity, wind speed, and cloudiness at noon each day. Then, they brought their poodle outside and recorded how quickly it sought the cool refuge of their air-conditioned classroom. They identified undesirable weather combinations and shared their findings with the local newspaper to help their community. During problem-based learning, students face a problem, think critically, gather more information, try out solutions, and share their results en route to learning both subject content and problem solving processes (e.g.,Yew & Goh, [<reflink idref="bib116" id="ref59">116</reflink>]).</p> <p>During <emph>inquiry-based learning</emph>, students act like scientists (Pedaste et al., [<reflink idref="bib78" id="ref60">78</reflink>]). For example, they wondered what affects poodles' discomfort on hot day? Perhaps heavier poodles suffered more? To test this idea, they gathered several poodles, measured the temperature at noon, brought them all outside, and recorded how quickly they went back inside. In short, they ask questions, generate hypotheses, design experiments, collect and analyze data, and accept or reject their hypotheses (Pedaste et al., [<reflink idref="bib78" id="ref61">78</reflink>]). Like problem-based learning, inquiry-based learning deals with real, relevant problems. The teacher scaffolds students, helping them make sense of their ideas, develop explanations based on evidence, and share their ideas (Hmelo-Silver et al., [<reflink idref="bib36" id="ref62">36</reflink>]). But unlike problem-based learning, inquiry-based learning pushes students into scientific discovery processes (Pedaste et al., [<reflink idref="bib78" id="ref63">78</reflink>]).</p> <p>In <emph>project-based learning</emph>, students take on bigger real-world challenges and often create a product (Hess et al., [<reflink idref="bib34" id="ref64">34</reflink>]), like making a doghouse for their poodle. These students discussed criteria for a comfortable doghouse, evaluated many doghouse designs before choosing one, bought materials, built the doghouse, and observed how their poodle used it. During such longer, personally meaningful projects, students have more autonomy to address complex issues, often needing support and input from peers and their teacher (Kokotsaki et al., [<reflink idref="bib44" id="ref65">44</reflink>]).</p> <p> <emph>Design-based learning</emph> is a subset of project-based learning in which students learn and apply theoretical knowledge to address larger problems and design solutions (Puente et al., [<reflink idref="bib81" id="ref66">81</reflink>]). For example, Ms. T asked groups of students to design cooling mats for their poodles. Scaffolded by Ms. T as needed, they researched mats and possible materials (esp. their <emph>heat transfer</emph>), specified design criteria, brainstormed, built prototypes, and tested them with their poodle. Then, they refined their cool mat design via cycles of redesign, building, testing, and evaluating. Lastly, they shared their best cooling mat with their class. During the design process, their teachers help students identify/clarify the problem; gather information about it; develop possible solutions; select the best one(s); construct prototype(s); and cycle through testing, evaluating, and revising, before explaining their final solution (Massachusetts, [<reflink idref="bib59" id="ref67">59</reflink>]).</p> <p>Many teachers struggle with projects and design activities, which are typically much longer and more complex than problem-based or inquiry-based learning activities. Such teachers often do not have enough STEM knowledge (Brophy et al., [<reflink idref="bib14" id="ref68">14</reflink>]; Dare et al., [<reflink idref="bib21" id="ref69">21</reflink>]), experience with project-based or design-based curricula (Frank et al., [<reflink idref="bib25" id="ref70">25</reflink>]; Nadelson et al., [<reflink idref="bib69" id="ref71">69</reflink>]), time, or curriculum flexibility (Helle et al., [<reflink idref="bib33" id="ref72">33</reflink>]). These hurdles obstruct them from helping their students complete these complex activities and learn from them (Lewis et al., [<reflink idref="bib49" id="ref73">49</reflink>]). By contrast, problem-based and inquiry-based learning are simpler to manage, pop up more often in classrooms (Albion, [<reflink idref="bib3" id="ref74">3</reflink>]), and yield more student learning (Panasan & Nuangchaler, [<reflink idref="bib76" id="ref75">76</reflink>]). Thus, we propose this hypothesis:</p> <p></p> <ulist> <item> H-3. Students in problem-based or inquiry-based lessons outperform students in project or design-based lessons during STEM integration.</item> </ulist> <hd id="AN0188452166-7">Engineering Design Skills</hd> <p>Scholars argue that integrating an iterative, <emph>engineering design process</emph> into K-12 STEM education helps students learn engineering design skills to tackle real-world problems: identify engineering constraints, propose engineering solutions, make decisions, and communicate effectively (e.g., Bryan et al., [<reflink idref="bib15" id="ref76">15</reflink>]; Guzey et al., [<reflink idref="bib30" id="ref77">30</reflink>]; Moore et al., [<reflink idref="bib66" id="ref78">66</reflink>]). These students learn to (a) link science and math through engineering principles and (b) crafting evidence-based engineering solutions for complex problems (Guzey et al., [<reflink idref="bib30" id="ref79">30</reflink>]). As scientific inquiry focuses on understanding phenomena and engineering design focuses on creating solutions (Ali & Tse, [<reflink idref="bib5" id="ref80">5</reflink>]), the Next Generation Science Standards (NGSS) in the United States urged their integration for greater resource and time efficiency (Dare et al., [<reflink idref="bib21" id="ref81">21</reflink>]; NRC, [<reflink idref="bib72" id="ref82">72</reflink>]). So, science teachers (unlike other teachers) did so for their curricula (Ross et al., [<reflink idref="bib87" id="ref83">87</reflink>]). However, many teachers do not have enough engineering design expertise to implement it (Hamad et al., [<reflink idref="bib32" id="ref84">32</reflink>]), which can reduce its impact on student learning. Thus, we propose this hypothesis:</p> <p></p> <ulist> <item> H-4. The STEM integration-achievement link is not stronger in interventions with engineering design skills.</item> </ulist> <hd id="AN0188452166-8">Cultural Values</hd> <p>Cultural values vary across countries. Some societies favor group interests over individual interests (<emph>collectivism</emph>), while others favor individual interests (<emph>individualism</emph>; LeTendre et al., [<reflink idref="bib48" id="ref85">48</reflink>]). In collectivist cultures, people pay more attention to one another, so they work together more effectively to attain group goals (e.g., collaborative learning with classmates; Chiu & Chow, [<reflink idref="bib18" id="ref86">18</reflink>]). Indeed, classmates' mean past achievement is linked to a student's current achievement more strongly in collectivist cultures than in individualist ones (Chiu & Chow, [<reflink idref="bib18" id="ref87">18</reflink>]). As STEM integration often includes group activities, classmates might collaborate more effectively and learn more in collectivist cultures than in individualist ones (e.g., McAtavey & Nikolovska, [<reflink idref="bib60" id="ref88">60</reflink>]; Tian et al., [<reflink idref="bib103" id="ref89">103</reflink>]). However, students in collectivist cultures avoid criticizing their peers to save face (Wong, [<reflink idref="bib113" id="ref90">113</reflink>]), which hinders their collaborative problem-solving (Wu et al., [<reflink idref="bib114" id="ref91">114</reflink>]). Hence, this remains an open question that we test in this study.</p> <p></p> <ulist> <item> H-5: The STEM integration-achievement link is stronger in collectivist cultures than individualist ones.</item> </ulist> <hd id="AN0188452166-9">Achievement Measures: Academic Subject and Standardized Tests</hd> <p>Achievement measures can differ across academic subjects and are not always standardized tests. Researchers often create custom tests to match their learning activities, so their greater coherence can yield stronger effect sizes. Conversely, using off-the-shelf tests that do not align as well with the activities can yield weaker effect sizes. For example, Fan and Yu ([<reflink idref="bib24" id="ref92">24</reflink>]) designed a STEM unit and a corresponding conceptual test to assess learners' understanding of mechanics, science, and math. Also, students more readily perceive, interact with, enjoy, learn from and remember activities that are more visible, accessible, and concrete (e.g., biology) rather than abstract ones (e.g., physics; Dong et al., [<reflink idref="bib22" id="ref93">22</reflink>]). Thus, we test whether measures of achievement content (STEM and academic subjects) are significant moderators.</p> <p>Unlike one-time, customized tests for a research study, some tests are psychometrically validated and standardized, with uniform administration, scoring, and interpretation procedures (e.g., National Assessment of Educational Progress [NAEP] or Programme for International Student Assessment [PISA]). NAEP assesses student achievement across subjects (NCES, [<reflink idref="bib73" id="ref94">73</reflink>]). PISA measures 15-year-olds' ability to use their reading, mathematics, and science knowledge and skills to meet real-life challenges (OECD, [<reflink idref="bib74" id="ref95">74</reflink>]). Hence, non-standardized tests like class tests are often more closely aligned than standardized tests to instruction and might yield larger intervention effect sizes (Li & Ma, [<reflink idref="bib51" id="ref96">51</reflink>]). Thus, we tested whether standardized (vs. unstandardized) tests are significant moderators.</p> <hd id="AN0188452166-10">Other Study Differences: Designs, Intervention Duration, and Participant Grade Levels</hd> <p>Following standard quality control protocols for meta-analyses, we also test for differences across study designs, intervention durations, and grade levels. Study designs included: (a) a controlled experiment with both pre- and post-tests (e.g., Hsiao et al., [<reflink idref="bib37" id="ref97">37</reflink>]), (b) only the experimental group's pre- and post-tests (which ignores informal learning across time, e.g., Pozarski Connolly, [<reflink idref="bib80" id="ref98">80</reflink>]), or (c) only post-tests of control and experimental groups (which ignores their prior learning, e.g., Capobianco et al., [<reflink idref="bib17" id="ref99">17</reflink>]. As controlled experiments likely show less bias (e.g., maturation bias, history bias; Kirk, [<reflink idref="bib43" id="ref100">43</reflink>]) than others (Torday & Baluška, [<reflink idref="bib104" id="ref101">104</reflink>]), study design might moderate the STEM integration-achievement link.</p> <p>Longer intervention duration allows for more learning time and better learning outcomes (e.g., Wahono et al. [[<reflink idref="bib109" id="ref102">109</reflink>]]). However, it also allows for more external events that interfere with learning and drive weaker results (Nahmias et al., [<reflink idref="bib70" id="ref103">70</reflink>]). Hence, the impact of intervention duration on learning is unclear.</p> <p>Past studies have not hypothesized how grade level (e.g., middle school vs. high school) might moderate the effect of STEM integration on learning outcomes. Still, following common meta-analysis practices, we test grade level as a moderator.</p> <hd id="AN0188452166-11">Present Study</hd> <p>This meta-analysis determines the overall link between STEM integration and student achievement. Furthermore, we test whether the following moderators account for past studies' different results: degree of STEM integration, learning activities, engineering design skills, cultural values, academic subject, standardized test, study design, intervention duration, and grade level.</p> <hd id="AN0188452166-12">Method</hd> <p></p> <hd id="AN0188452166-13">Literature Search Procedure</hd> <p>We used the PRISMA guidelines in our literature search and selection processes (see Fig. 1, Moher et al., [<reflink idref="bib65" id="ref104">65</reflink>]). To identify original studies up to January 8, 2024 inclusive, we searched six electronic databases: Education Resources Information Center (ERIC), PsychINFO, JSTOR, Web of Science, Institute of Electrical and Electronics Engineers (IEEE) Xplore, ProQuest Dissertations and Theses. Our searches used these keyword combinations: (<emph>student success, student achievement, academic achievement, or student performance), AND (STEM integration,</emph> or <emph>integrated STEM)</emph>. We checked the references of published studies of STEM integration but did not find additional original studies. Then, we removed duplicates.</p> <p>Graph: Fig. 1 PRISMA 2009 Flow Diagram (Moher et al., [<reflink idref="bib65" id="ref105">65</reflink>])</p> <hd id="AN0188452166-14">Inclusion Criteria</hd> <p>We included articles that met all six of the following criteria (see Table 1). First, the study was related to STEM integration. Second, the study reported at least one measured student achievement outcome. Third, the study was a controlled experiment (post-tests only or both pre- and post-tests) or a treatment only condition with both pre- and post-tests. Fourth, the study reported means, standard deviations, t values, F values, or other sufficient information to compute effect sizes. Fifth, the study specified the type of STEM integration (e.g., Olivarez, [<reflink idref="bib75" id="ref106">75</reflink>]; Williams, [<reflink idref="bib112" id="ref107">112</reflink>]). Our final data set has 79 effect sizes from 40 studies.</p> <p>Table 1 Studies Included in the Meta-analysis</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Study</p></th><th align="left"><p>Effect size (<italic>g</italic>)</p></th><th align="left"><p>Degree of integration</p></th><th align="left"><p>Learning</p><p>Activity<sup>a</sup></p></th><th align="left"><p>Engineering design skill</p></th><th align="left"><p>Intervention</p><p>Duration</p></th><th align="left"><p>Cultural values<sup>b</sup></p></th><th align="left"><p>Academic subject<sup>c</sup></p></th><th align="left"><p>Grade</p><p>Level<sup>d</sup></p></th><th align="left"><p>Study Design<sup>e</sup></p></th><th align="left"><p>Standardized test or not<sup>f</sup></p></th></tr></thead><tbody><tr><td align="left"><p>Acar et al., <xref ref-type="bibr" rid="bibr1">2018</xref></p></td><td char="." align="char"><p>1.958</p></td><td align="left"><p>Content</p></td><td align="left"><p>NA</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Acar et al., <xref ref-type="bibr" rid="bibr1">2018</xref></p></td><td char="." align="char"><p>1.473</p></td><td align="left"><p>Content</p></td><td align="left"><p>NA</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Acar et al., <xref ref-type="bibr" rid="bibr1">2018</xref></p></td><td char="." align="char"><p>0.299</p></td><td align="left"><p>Content</p></td><td align="left"><p>NA</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Acar et al., <xref ref-type="bibr" rid="bibr1">2018</xref></p></td><td char="." align="char"><p>1.556</p></td><td align="left"><p>Content</p></td><td align="left"><p>NA</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Akkaya & Benzer, <xref ref-type="bibr" rid="bibr2">2020</xref></p></td><td char="." align="char"><p>2.044</p></td><td align="left"><p>Content</p></td><td align="left"><p>NA</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Alameh, <xref ref-type="bibr" rid="bibr4">2018</xref></p></td><td char="." align="char"><p>0.379</p></td><td align="left"><p>Content</p></td><td align="left"><p>3</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Alameh, <xref ref-type="bibr" rid="bibr4">2018</xref></p></td><td char="." align="char"><p>0.051</p></td><td align="left"><p>Content</p></td><td align="left"><p>3</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Angwal et al., <xref ref-type="bibr" rid="bibr8">2019</xref></p></td><td char="." align="char"><p>1.650</p></td><td align="left"><p>Context</p></td><td align="left"><p>mix</p></td><td align="left"><p>Yes</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Anwar et al., <xref ref-type="bibr" rid="bibr7">2022</xref></p></td><td char="." align="char"><p>0.742</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p> ≤ 4 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Capobianco et al., <xref ref-type="bibr" rid="bibr17">2021</xref></p></td><td char="." align="char"><p>1.656</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> ≤ 4 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Capobianco et al., <xref ref-type="bibr" rid="bibr17">2021</xref></p></td><td char="." align="char"><p>1.514</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> ≤ 4 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Cotabish et al., <xref ref-type="bibr" rid="bibr20">2013</xref></p></td><td char="." align="char"><p>1.192</p></td><td align="left"><p>Content</p></td><td align="left"><p>3</p></td><td align="left"><p>No</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Fan & Yu, <xref ref-type="bibr" rid="bibr24">2017</xref></p></td><td char="." align="char"><p>1.199</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>STEM</p></td><td align="left"><p>High</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Gazibeyoglu & Aydin, <xref ref-type="bibr" rid="bibr26">2019</xref></p></td><td char="." align="char"><p>0.934</p></td><td align="left"><p>Context</p></td><td align="left"><p>3</p></td><td align="left"><p>No</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>2</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>0.261</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Engineer</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>0.047</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>0.381</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>0.566</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>0.320</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>0.136</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>−0.383</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Element.</p></td><td align="left"><p>2</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>0.233</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Engineer</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>0.128</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>0.752</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>0.236</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>0.315</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>0.517</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Guzey et al., <xref ref-type="bibr" rid="bibr29">2017</xref></p></td><td char="." align="char"><p>−0.204</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Element.</p></td><td align="left"><p>2</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Hsiao et al., <xref ref-type="bibr" rid="bibr37">2017</xref></p></td><td char="." align="char"><p>0.466</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> ≤ 4 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>STEM</p></td><td align="left"><p>High</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p> Izgi & Kalayci, <xref ref-type="bibr" rid="bibr38">2020</xref></p></td><td char="." align="char"><p>1.326</p></td><td align="left"><p>Context</p></td><td align="left"><p>3</p></td><td align="left"><p>No</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>1</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Jahan, <xref ref-type="bibr" rid="bibr39">2018</xref></p></td><td char="." align="char"><p>0.018</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>3</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Jahan, <xref ref-type="bibr" rid="bibr39">2018</xref></p></td><td char="." align="char"><p>0.241</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>3</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Jahan, <xref ref-type="bibr" rid="bibr39">2018</xref></p></td><td char="." align="char"><p>0.640</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>3</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>James, <xref ref-type="bibr" rid="bibr40">2014</xref></p></td><td char="." align="char"><p>0.297</p></td><td align="left"><p>Content</p></td><td align="left"><p>1</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Middle</p></td><td align="left"><p>2</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>James, <xref ref-type="bibr" rid="bibr40">2014</xref></p></td><td char="." align="char"><p>0.443</p></td><td align="left"><p>Content</p></td><td align="left"><p>1</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>2</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Kağnıcı & Sadi, <xref ref-type="bibr" rid="bibr41">2021</xref></p></td><td char="." align="char"><p>1.103</p></td><td align="left"><p>Context</p></td><td align="left"><p>3</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Kelley et al., <xref ref-type="bibr" rid="bibr42">2023</xref></p></td><td char="." align="char"><p>0.470</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>STEM</p></td><td align="left"><p>High</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Kurt & Benzer, <xref ref-type="bibr" rid="bibr45">2020</xref></p></td><td char="." align="char"><p>1.489</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>No</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Li et al., <xref ref-type="bibr" rid="bibr52">2016</xref></p></td><td char="." align="char"><p>0.037</p></td><td align="left"><p>Context</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Li et al., <xref ref-type="bibr" rid="bibr50">2018</xref></p></td><td char="." align="char"><p>0.364</p></td><td align="left"><p>Tool</p></td><td align="left"><p>2</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>High</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Li et al., <xref ref-type="bibr" rid="bibr50">2018</xref></p></td><td char="." align="char"><p>0.513</p></td><td align="left"><p>Tool</p></td><td align="left"><p>2</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Lie et al., <xref ref-type="bibr" rid="bibr55">2019</xref></p></td><td char="." align="char"><p>0.140</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Engineer</p></td><td align="left"><p>Element.</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Lie et al., <xref ref-type="bibr" rid="bibr55">2019</xref></p></td><td char="." align="char"><p>0.246</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Engineer</p></td><td align="left"><p>Middle</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Lie et al., <xref ref-type="bibr" rid="bibr55">2019</xref></p></td><td char="." align="char"><p>0.228</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Engineer</p></td><td align="left"><p>Middle</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Lie et al., <xref ref-type="bibr" rid="bibr55">2019</xref></p></td><td char="." align="char"><p>0.045</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Engineer</p></td><td align="left"><p>Middle</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Lippert & Seals, <xref ref-type="bibr" rid="bibr56">2023</xref></p></td><td char="." align="char"><p>0.170</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Middle</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>McClain, <xref ref-type="bibr" rid="bibr61">2015</xref></p></td><td char="." align="char"><p>0.260</p></td><td align="left"><p>Context</p></td><td align="left"><p>2</p></td><td align="left"><p>No</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>McGehee, <xref ref-type="bibr" rid="bibr61">2015</xref></p></td><td char="." align="char"><p>0.218</p></td><td align="left"><p>NA</p></td><td align="left"><p>2</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>High</p></td><td align="left"><p>2</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>McGehee, <xref ref-type="bibr" rid="bibr61">2015</xref></p></td><td char="." align="char"><p>−0.055</p></td><td align="left"><p>NA</p></td><td align="left"><p>2</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>2</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>McGehee, <xref ref-type="bibr" rid="bibr61">2015</xref></p></td><td char="." align="char"><p>−0.069</p></td><td align="left"><p>NA</p></td><td align="left"><p>2</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>High</p></td><td align="left"><p>2</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>McGehee, <xref ref-type="bibr" rid="bibr61">2015</xref></p></td><td char="." align="char"><p>−0.153</p></td><td align="left"><p>NA</p></td><td align="left"><p>2</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>2</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>McHugh, <xref ref-type="bibr" rid="bibr63">2016</xref></p></td><td char="." align="char"><p>1.876</p></td><td align="left"><p>Content</p></td><td align="left"><p>3</p></td><td align="left"><p>No</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>McHugh, <xref ref-type="bibr" rid="bibr63">2016</xref></p></td><td char="." align="char"><p>−0.221</p></td><td align="left"><p>Content</p></td><td align="left"><p>3</p></td><td align="left"><p>No</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>2</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Parlakay & Koç, <xref ref-type="bibr" rid="bibr77">2020</xref></p></td><td char="." align="char"><p>0.559</p></td><td align="left"><p>Context</p></td><td align="left"><p>2</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Pozarski Connolly, <xref ref-type="bibr" rid="bibr80">2017</xref></p></td><td char="." align="char"><p>1.266</p></td><td align="left"><p>Content</p></td><td align="left"><p>3</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Pozarski Connolly, <xref ref-type="bibr" rid="bibr80">2017</xref></p></td><td char="." align="char"><p>0.669</p></td><td align="left"><p>Content</p></td><td align="left"><p>3</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Rehmat & Hartley, <xref ref-type="bibr" rid="bibr82">2020</xref></p></td><td char="." align="char"><p>2.275</p></td><td align="left"><p>Context</p></td><td align="left"><p>1</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>STEM</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Rehmat & Hartley, <xref ref-type="bibr" rid="bibr82">2020</xref></p></td><td char="." align="char"><p>1.294</p></td><td align="left"><p>Context</p></td><td align="left"><p>1</p></td><td align="left"><p>Yes</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>STEM</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Robinson, <xref ref-type="bibr" rid="bibr83">2016</xref></p></td><td char="." align="char"><p>0.536</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Middle</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Robison et al., <xref ref-type="bibr" rid="bibr84">2014</xref></p></td><td char="." align="char"><p>1.513</p></td><td align="left"><p>Content</p></td><td align="left"><p>3</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Robison et al., <xref ref-type="bibr" rid="bibr84">2014</xref></p></td><td char="." align="char"><p>2.616</p></td><td align="left"><p>Content</p></td><td align="left"><p>3</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Robison et al., <xref ref-type="bibr" rid="bibr84">2014</xref></p></td><td char="." align="char"><p>2.230</p></td><td align="left"><p>Content</p></td><td align="left"><p>3</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Sabag & Trotskovsky, <xref ref-type="bibr" rid="bibr89">2013</xref></p></td><td char="." align="char"><p>0.533</p></td><td align="left"><p>Content</p></td><td align="left"><p>NA</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Math</p></td><td align="left"><p>High</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Sarican & Akgunduz, <xref ref-type="bibr" rid="bibr91">2018</xref></p></td><td char="." align="char"><p>0.381</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Sauder, <xref ref-type="bibr" rid="bibr92">2023</xref></p></td><td char="." align="char"><p>0.802</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p> ≤ 4 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Shelden, <xref ref-type="bibr" rid="bibr94">2021</xref></p></td><td char="." align="char"><p>0.044</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Element.</p></td><td align="left"><p>3</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Shelden, <xref ref-type="bibr" rid="bibr94">2021</xref></p></td><td char="." align="char"><p>0.159</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>High</p></td><td align="left"><p>3</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Srihongsa et al., <xref ref-type="bibr" rid="bibr97">2017</xref></p></td><td char="." align="char"><p>2.150</p></td><td align="left"><p>Content</p></td><td align="left"><p>NA</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Element.</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Stitham, <xref ref-type="bibr" rid="bibr98">2018</xref></p></td><td char="." align="char"><p>0.466</p></td><td align="left"><p>Content</p></td><td align="left"><p>mix</p></td><td align="left"><p>No</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Sunyoung et al., <xref ref-type="bibr" rid="bibr101">2016</xref></p></td><td char="." align="char"><p>0.351</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>No</p></td><td align="left"><p> > 8 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Math</p></td><td align="left"><p>High</p></td><td align="left"><p>1</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Tati et al., <xref ref-type="bibr" rid="bibr102">2017</xref></p></td><td char="." align="char"><p>0.801</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>2</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Tati et al., <xref ref-type="bibr" rid="bibr102">2017</xref></p></td><td char="." align="char"><p>0.876</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>T & Engineer</p></td><td align="left"><p>Middle</p></td><td align="left"><p>2</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Tati et al., <xref ref-type="bibr" rid="bibr102">2017</xref></p></td><td char="." align="char"><p>0.820</p></td><td align="left"><p>Content</p></td><td align="left"><p>2</p></td><td align="left"><p>Yes</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Math</p></td><td align="left"><p>Middle</p></td><td align="left"><p>2</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Tsai et al., <xref ref-type="bibr" rid="bibr105">2021</xref></p></td><td char="." align="char"><p>0.547</p></td><td align="left"><p>Content</p></td><td align="left"><p>3</p></td><td align="left"><p>No</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Vallera & Bodzin <xref ref-type="bibr" rid="bibr106">2020</xref></p></td><td char="." align="char"><p>1.136</p></td><td align="left"><p>Context</p></td><td align="left"><p>2</p></td><td align="left"><p>No</p></td><td align="left"><p> ≤ 4 weeks</p></td><td align="left"><p>Indiv.</p></td><td align="left"><p>STEM</p></td><td align="left"><p>Element.</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Yaki et al., <xref ref-type="bibr" rid="bibr115">2019</xref></p></td><td char="." align="char"><p>0.329</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Yaki et al., <xref ref-type="bibr" rid="bibr115">2019</xref></p></td><td char="." align="char"><p>1.277</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Yaki et al., <xref ref-type="bibr" rid="bibr115">2019</xref></p></td><td char="." align="char"><p>1.596</p></td><td align="left"><p>Content</p></td><td align="left"><p>4</p></td><td align="left"><p>Yes</p></td><td align="left"><p>4–8 weeks</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>High</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr><tr><td align="left"><p>Hasançebi et al., <xref ref-type="bibr" rid="bibr31">2021</xref></p></td><td char="." align="char"><p>0.795</p></td><td align="left"><p>NA</p></td><td align="left"><p>3</p></td><td align="left"><p>No</p></td><td align="left"><p>NA</p></td><td align="left"><p>Collect.</p></td><td align="left"><p>Science</p></td><td align="left"><p>Middle</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td></tr></tbody></table> </ephtml> </p> <p> <emph>Note.</emph> <sups>a</sups>1 = problem-based learning, 2 = project-based learning, 3 = inquiry-based learning, 4 = design-based learning; <sups>b</sups> Collect. = Collectivism, Indiv. = Individualism.<sups>c</sups>T = technology, Engineer = engineering; <sups>d</sups> Element. = elementary school, middle = middle school, high = high school; <sups>e</sups>1 = Pre-post experimental and control, 2 = Control and experimental group's posts, 3 = Experimental group's pre and post; <sups>f</sups> T = technology, Engineer = engineering;<sups>d</sups>1 = standardized, 2 = non-standardized</p> <hd id="AN0188452166-15">Coding</hd> <p>Three authors coded the included studies according to the above moderators: author, learning activity (problem-based learning, project-based learning, inquiry-based learning, and design-based learning), intervention duration(≤ 4 weeks, 4–8 weeks, > 8 weeks), degree of integration (content integration, context integration, and tool integration), academic subjects [e.g., science, math, engineering design, STEM], standardized test (or non-standardized), cultural values (collectivist vs. individualist), engineering design skill (yes or no), and grade level (elementary school, middle school, high school). See Table 1. Although most studies explicitly specified the learning activity as either problem-based learning, project-based learning, inquiry-based learning, or design-based learning, some studies used other terminology, so we carefully read their definitions and coded them accordingly (e.g., Angwal et al.'s [[<reflink idref="bib8" id="ref108">8</reflink>]] 5E model is a type of inquiry-based learning). Notably, if a study included more than one subject, we coded it as STEM. Also, many of these interventions covered a few lessons (days), a science unit (four or more weeks), or a semester (or longer), so we categorized interventions into three categories: less than 4 weeks, 4–8 weeks, and more than 8 weeks.</p> <p>Initially, three authors independently coded one study in the first week, another two studies in the second week, and another two studies in the third week. Each week, they discussed coding discrepancies, resolved the disagreement by consensus, and specified/updated definitions of codes. Then, they independently coded randomly assigned studies. Next, the other two authors checked each author's independent coding. These three authors resolved all disagreements through discussion and consensus.</p> <hd id="AN0188452166-16">Calculation of Effect Size</hd> <p>We used the effect size Hedge's <emph>g</emph>. For studies with only the experiment group's pre- and post-test scores, we computed its standardized mean difference of test scores before and after STEM intervention: <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi><mo>=</mo><mfenced close=")" open="("><mn>1</mn><mo>-</mo><mfrac><mn>3</mn><mrow><mn>4</mn><mi>d</mi><mi>f</mi><mo>-</mo><mn>1</mn></mrow></mfrac></mfenced><mi>d</mi></mrow></math> </ephtml> , with <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>d</mi><mo>=</mo><mfrac><mrow><msub><mover><mi>x</mi><mo>¯</mo></mover><mrow><mi mathvariant="italic">post</mi></mrow></msub><mo>-</mo><msub><mover><mi>x</mi><mo>¯</mo></mover><mrow><mi mathvariant="italic">pre</mi></mrow></msub></mrow><msub><mi>S</mi><mrow><mi mathvariant="italic">within</mi></mrow></msub></mfrac></mrow></math> </ephtml> , <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>S</mi><mrow><mi mathvariant="italic">within</mi></mrow></msub><mo>=</mo><mfrac><msub><mi>S</mi><mrow><mi mathvariant="italic">diff</mi></mrow></msub><msqrt><mrow><mn>2</mn><mfenced close=")" open="("><mn>1</mn><mo>-</mo><mi>r</mi></mfenced></mrow></msqrt></mfrac></mrow></math> </ephtml> .</p> <p>For studies with only post-test scores of the experiment and control groups,</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi><mo>=</mo><mfenced close=")" open="("><mn>1</mn><mo>-</mo><mfrac><mn>3</mn><mrow><mn>4</mn><mi>d</mi><mi>f</mi><mo>-</mo><mn>1</mn></mrow></mfrac></mfenced><mi>d</mi></mrow></math> </ephtml> , where <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>d</mi><mo>=</mo><mfrac><mrow><msub><mover><mi>x</mi><mo>¯</mo></mover><mn>1</mn></msub><mo>-</mo><msub><mover><mi>x</mi><mo>¯</mo></mover><mn>2</mn></msub></mrow><msub><mi>S</mi><mrow><mi mathvariant="italic">within</mi></mrow></msub></mfrac></mrow></math> </ephtml> , <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>S</mi><mrow><mi mathvariant="italic">within</mi></mrow></msub><mo>=</mo><msqrt><mfrac><mrow><mfenced close=")" open="("><msub><mi>n</mi><mn>1</mn></msub><mo>-</mo><mn>1</mn></mfenced><msubsup><mi>S</mi><mrow><mn>1</mn></mrow><mn>2</mn></msubsup><mo>+</mo><mrow><mo stretchy="false">(</mo><msub><mi>n</mi><mn>2</mn></msub><mo>-</mo><mn>1</mn><mo stretchy="false">)</mo></mrow><msubsup><mi>S</mi><mrow><mn>2</mn></mrow><mn>2</mn></msubsup></mrow><mrow><msub><mi>n</mi><mn>1</mn></msub><mo>+</mo><msub><mi>n</mi><mn>2</mn></msub><mo>-</mo><mn>2</mn></mrow></mfrac></msqrt></mrow></math> </ephtml> , with the two groups' respective sample means <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mover><mi>x</mi><mo>¯</mo></mover><mn>1</mn></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mover><mi>x</mi><mo>¯</mo></mover><mn>2</mn></msub></math> </ephtml> , standard deviations <emph>S</emph><subs>1</subs> and <emph>S</emph><subs>2</subs>, and sample sizes n<subs>1</subs> and n<subs>2</subs> (Borenstein et al., [<reflink idref="bib13" id="ref109">13</reflink>]).</p> <p>For studies with both pre- and post-test scores of the experiment and control groups,</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi><mo>=</mo><mfenced close=")" open="("><mn>1</mn><mo>-</mo><mfrac><mn>3</mn><mrow><mn>4</mn><mi>d</mi><mi>f</mi><mo>-</mo><mn>1</mn></mrow></mfrac></mfenced><mi>d</mi></mrow></math> </ephtml> , where <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>d</mi><mo>=</mo><mfrac><mrow><mrow><mo stretchy="false">(</mo><msub><mover><mi>x</mi><mo>¯</mo></mover><mrow><mn>1</mn><mi>_</mi><mi>p</mi><mi>o</mi><mi>s</mi><mi>t</mi></mrow></msub><mo>-</mo><msub><mover><mi>x</mi><mo>¯</mo></mover><mrow><mn>1</mn><mi>_</mi><mi>p</mi><mi>r</mi><mi>e</mi></mrow></msub><mo stretchy="false">)</mo></mrow><mo>-</mo><mrow><mo stretchy="false">(</mo><msub><mover><mi>x</mi><mo>¯</mo></mover><mrow><mn>2</mn><mi>_</mi><mi>p</mi><mi>o</mi><mi>s</mi><mi>t</mi></mrow></msub><mo>-</mo><msub><mover><mi>x</mi><mo>¯</mo></mover><mrow><mn>2</mn><mi>_</mi><mi>p</mi><mi>r</mi><mi>e</mi></mrow></msub><mo stretchy="false">)</mo></mrow></mrow><msub><mrow><mi mathvariant="italic">SD</mi></mrow><mrow><mi mathvariant="italic">post</mi></mrow></msub></mfrac></mrow></math> </ephtml> , and</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mi mathvariant="italic">SD</mi></mrow><mrow><mi mathvariant="italic">post</mi></mrow></msub><mo>=</mo><msqrt><mfrac><mrow><mfenced close=")" open="("><msub><mi>n</mi><mrow><mn>2</mn><mi>_</mi><mi>p</mi><mi>o</mi><mi>s</mi><mi>t</mi></mrow></msub><mo>-</mo><mn>1</mn></mfenced><msubsup><mi>S</mi><mrow><mn>2</mn><mi>_</mi><mi>p</mi><mi>o</mi><mi>s</mi><mi>t</mi></mrow><mn>2</mn></msubsup><mo>+</mo><mrow><mo stretchy="false">(</mo><msub><mi>n</mi><mrow><mn>1</mn><mi>_</mi><mi>p</mi><mi>o</mi><mi>s</mi><mi>t</mi></mrow></msub><mo>-</mo><mn>1</mn><mo stretchy="false">)</mo></mrow><msubsup><mi>S</mi><mrow><mn>1</mn><mi>_</mi><mi>p</mi><mi>o</mi><mi>s</mi><mi>t</mi></mrow><mn>2</mn></msubsup></mrow><mrow><msub><mi>n</mi><mrow><mn>2</mn><mi>_</mi><mi>p</mi><mi>o</mi><mi>s</mi><mi>t</mi></mrow></msub><mo>+</mo><msub><mi>n</mi><mrow><mn>1</mn><mi>_</mi><mi>p</mi><mi>o</mi><mi>s</mi><mi>t</mi></mrow></msub><mo>-</mo><mn>2</mn></mrow></mfrac></msqrt></mrow></math> </ephtml> , with the respective mean scores of the pretests and posttests of the experimental group <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover><mi>x</mi><mo>¯</mo></mover><mrow><mn>1</mn><mi>%5f</mi><mi>p</mi><mi>r</mi><mi>e</mi></mrow></msub><mspace width="0.277778em" /><mi>a</mi><mi>n</mi><mi>d</mi><mspace width="0.277778em" /><msub><mover><mi>x</mi><mo>¯</mo></mover><mrow><mn>1</mn><mi>_</mi><mi>p</mi><mi>o</mi><mi>s</mi><mi>t</mi></mrow></msub></mrow></math> </ephtml> and of the control group <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover><mi>x</mi><mo>¯</mo></mover><mrow><mn>2</mn><mi>%5f</mi><mi>p</mi><mi>r</mi><mi>e</mi></mrow></msub><mspace width="0.277778em" /><mi>a</mi><mi>n</mi><mi>d</mi><mspace width="0.277778em" /><msub><mover><mi>x</mi><mo>¯</mo></mover><mrow><mn>2</mn><mi>_</mi><mi>p</mi><mi>o</mi><mi>s</mi><mi>t</mi></mrow></msub></mrow></math> </ephtml> (Sung et al., [<reflink idref="bib100" id="ref110">100</reflink>]).</p> <p>We used the inverse-variance weights to calculate the overall effect size. For studies with two or more control groups, we treated them independently. If a study was published both as a journal article and as a dissertation, we used the journal article results.</p> <hd id="AN0188452166-17">Statistical Analyses</hd> <p>We analyzed these data with Comprehensive Meta Analysis software (CMA 3.0). As these studies used a wide range of STEM intervention approaches and sample types, we used a random effects model with inverse-variance weights to calculate the mean effect size (Borenstein et al., [<reflink idref="bib12" id="ref111">12</reflink>]). A statistically significant <emph>Q</emph> statistic or large <emph>I</emph><sups><emph>2</emph></sups> indicates effect size heterogeneity (Higgins et al., [<reflink idref="bib35" id="ref112">35</reflink>]).</p> <p>We tested for publication bias with a funnel plot, Egger's regression, and fail-safe number (N<subs>fs</subs>). The following results would suggest a low risk of publication bias: (a) effect sizes are mostly within the funnel and show axis symmetry; (b) Egger's linear regression intercept is not significant and near-zero; and (c) N<subs>fs</subs> > 5<emph> k</emph> + 10 (<emph>k</emph> = number of original studies, Rothstein et al., [<reflink idref="bib88" id="ref113">88</reflink>]).</p> <hd id="AN0188452166-18">Results</hd> <p></p> <hd id="AN0188452166-19">Overall Effect Size</hd> <p>STEM integration has a positive effect on student achievement (<emph>g</emph> = 0.661, 95% CI [0.548,0.774], <emph>p</emph> < 0.001), supporting H-1 (students in STEM integration activities outperform other students on learning outcomes). The effect size varied significantly across studies (<emph>Q</emph> [<reflink idref="bib78" id="ref114">78</reflink>] = 2,954, <emph>p</emph> < 0.001; <emph>I</emph><sups>2</sups> = 97,<emph> τ</emph><sups><emph>2</emph></sups> = 0.222, see Table 2). See the forest plot in Fig. 2.</p> <p>Table 2 Random Model of the STEM Integration's Effect on Student STEM Achievement</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p>Weighted mean <italic>g</italic></p></th><th align="left" rowspan="2"><p><italic>k</italic></p></th><th align="left" colspan="2"><p>Effect size and 95% interval</p></th><th align="left" /><th align="left"><p>Test of null</p><p>(2-Tail)</p></th><th align="left" /><th align="left" colspan="2"><p>Heterogeneity</p></th><th align="left" /><th align="left" colspan="3"><p>Tau-squared</p></th></tr><tr><th align="left"><p>LL</p></th><th align="left"><p>UL</p></th><th align="left" /><th align="left"><p>z-Value</p></th><th align="left" /><th align="left"><p><italic>Q</italic>-Value</p></th><th align="left"><p>I-squared</p></th><th align="left" /><th align="left"><p>Tau Squared</p></th><th align="left"><p>SE</p></th><th align="left"><p>Tau</p></th></tr></thead><tbody><tr><td align="left"><p>0.661</p></td><td align="left"><p>79</p></td><td char="." align="char"><p>0.548</p></td><td char="." align="char"><p>0.774</p></td><td align="left" /><td char="." align="char"><p>11.463*</p></td><td align="left" /><td char="." align="char"><p>2954.147<sup>*</sup></p></td><td char="." align="char"><p>97.360</p></td><td align="left" /><td char="." align="char"><p>0.222</p></td><td char="." align="char"><p>0.073</p></td><td char="." align="char"><p>0.471</p></td></tr></tbody></table> </ephtml> </p> <p> <emph>Note</emph>. <sups>*</sups><emph>p</emph> <.001</p> <p>Graph: Fig. 2 Forest Plot for the Random-effects Model. Note: This figure is an output from CMA software. CMA uses "a", "b", "c" etc., after the authors to differentiate different data used in the same study</p> <hd id="AN0188452166-20">Publication Bias</hd> <p>The funnel plot, Egger's regression, and N<subs>fs</subs> showed no evidence of publication bias. The effect sizes were mostly within their funnel without any sharp asymmetry (see Fig. 3). Egger's regression showed a non-significant intercept value of 1.429 (95% CI [−0.606, 3.464], <emph>p</emph> > 0.05). N<subs>fs</subs> (<reflink idref="bib40" id="ref115">40</reflink>,<reflink idref="bib437" id="ref116">437</reflink>) far exceeded its respective threshold (405 = 5*79 + 10).</p> <p>Graph: Fig. 3 Funnel Plot of the Effect Sizes of STEM Integration on Students' Academic Achievement</p> <hd id="AN0188452166-21">Moderator</hd> <p></p> <hd id="AN0188452166-22">Degree of STEM Integration</hd> <p>The degree of integration moderated the link between STEM integration and achievement (<emph>Q</emph><subs>between</subs> = 7.227, <emph>df</emph> = 2, <emph>p</emph> < 0.05, see Table 3). Specifically, context integration yielded the largest effect size (<emph>g</emph> = 1.063, 95% CI [0.668, 1.457], <emph>p</emph> < 0.001) followed content integration (<emph>g</emph> = 0.654, 95% CI [0.528, 0.780], <emph>p</emph> < 0.001) and then tool integration (<emph>g</emph> = 0.441, 95% CI [0.207, 0.676], <emph>p</emph> < 0.001), supporting H-2 (significant differences in integrations: context vs. content vs. tool).</p> <p>Table 3 The Effect of STEM Integration on Student STEM Achievement: Univariate Analysis of Variance for Moderator Variables</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p>Moderator</p></th><th align="left" rowspan="2"><p><italic>Q</italic><sub>between</sub></p></th><th align="left" rowspan="2"><p><italic>k</italic></p></th><th align="left" rowspan="2"><p>SE</p></th><th align="left" rowspan="2"><p>Weighted mean <italic>g</italic></p></th><th align="left" colspan="2"><p>95% CI</p></th><th align="left" rowspan="2"><p><italic>Q</italic><sub>within</sub></p></th><th align="left" rowspan="2"><p><italic>I</italic><sup>2</sup></p></th><th align="left" rowspan="2"><p><italic>τ</italic><sup><italic>2</italic></sup></p></th></tr><tr><th align="left"><p>LL</p></th><th align="left"><p>UL</p></th></tr></thead><tbody><tr><td align="left"><p><underline>Degree of Integration</underline></p></td><td char="." align="char"><p>7.227<sup>*</sup></p></td><td align="left" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td align="left" /><td char="." align="char" /><td char="." align="char" /></tr><tr><td align="left"><p>Content integration</p></td><td char="." align="char" /><td align="left"><p>62</p></td><td char="." align="char"><p>0.064</p></td><td char="." align="char"><p>0.654<sup>*</sup></p></td><td char="." align="char"><p>0.528</p></td><td char="." align="char"><p>0.780</p></td><td align="left"><p>2794.903<sup>*</sup></p></td><td char="." align="char"><p>97.817</p></td><td char="." align="char"><p>0.219</p></td></tr><tr><td align="left"><p>Context integration</p></td><td char="." align="char" /><td align="left"><p>10</p></td><td char="." align="char"><p>0.201</p></td><td char="." align="char"><p>1.063<sup>*</sup></p></td><td char="." align="char"><p>0.668</p></td><td char="." align="char"><p>1.457</p></td><td align="left"><p>55.459<sup>*</sup></p></td><td char="." align="char"><p>83.772</p></td><td char="." align="char"><p>0.335</p></td></tr><tr><td align="left"><p>Tool integration</p></td><td char="." align="char" /><td align="left"><p>2</p></td><td char="." align="char"><p>0.120</p></td><td char="." align="char"><p>0.441<sup>*</sup></p></td><td char="." align="char"><p>0.207</p></td><td char="." align="char"><p>0.676</p></td><td align="left"><p>0.388</p></td><td char="." align="char"><p>0.000</p></td><td char="." align="char"><p>0.000</p></td></tr><tr><td align="left"><p><underline>Learning Activities</underline></p></td><td char="." align="char"><p>21.347<sup>*</sup></p></td><td align="left" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td align="left" /><td char="." align="char" /><td char="." align="char" /></tr><tr><td align="left"><p>Problem-based learning</p></td><td char="." align="char" /><td align="left"><p>4</p></td><td char="." align="char"><p>0.194</p></td><td char="." align="char"><p>0.941<sup>*</sup></p></td><td char="." align="char"><p>0.561</p></td><td char="." align="char"><p>1.321</p></td><td align="left"><p>75.576<sup>*</sup></p></td><td char="." align="char"><p>96.030</p></td><td char="." align="char"><p>0.126</p></td></tr><tr><td align="left"><p>Project-based learning</p></td><td char="." align="char" /><td align="left"><p>23</p></td><td char="." align="char"><p>0.074</p></td><td char="." align="char"><p>0.401<sup>*</sup></p></td><td char="." align="char"><p>0.255</p></td><td char="." align="char"><p>0.547</p></td><td align="left"><p>352.342<sup>*</sup></p></td><td char="." align="char"><p>93.756</p></td><td char="." align="char"><p>0.098</p></td></tr><tr><td align="left"><p>Inquiry-based learning</p></td><td char="." align="char" /><td align="left"><p>15</p></td><td char="." align="char"><p>0.152</p></td><td char="." align="char"><p>1.051<sup>*</sup></p></td><td char="." align="char"><p>0.754</p></td><td char="." align="char"><p>1.349</p></td><td align="left"><p>752.728<sup>*</sup></p></td><td char="." align="char"><p>98.140</p></td><td char="." align="char"><p>0.291</p></td></tr><tr><td align="left"><p>Design-based learning</p></td><td char="." align="char" /><td align="left"><p>28</p></td><td char="." align="char"><p>0.075</p></td><td char="." align="char"><p>0.442<sup>*</sup></p></td><td char="." align="char"><p>0.294</p></td><td char="." align="char"><p>0.589</p></td><td align="left"><p>571.077<sup>*</sup></p></td><td char="." align="char"><p>95.272</p></td><td char="." align="char"><p>0.130</p></td></tr><tr><td align="left"><p><underline>Involving engineering design skills or not</underline></p></td><td char="." align="char"><p>0.018</p></td><td align="left" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td align="left" /><td char="." align="char" /><td char="." align="char" /></tr><tr><td align="left"><p>Not involving</p></td><td char="." align="char" /><td align="left"><p>27</p></td><td char="." align="char"><p>0.100</p></td><td char="." align="char"><p>0.650<sup>*</sup></p></td><td char="." align="char"><p>0.454</p></td><td char="." align="char"><p>0.847</p></td><td align="left"><p>510.743<sup>*</sup></p></td><td char="." align="char"><p>94.909</p></td><td char="." align="char"><p>0.221</p></td></tr><tr><td align="left"><p>Involving</p></td><td char="." align="char" /><td align="left"><p>52</p></td><td char="." align="char"><p>0.072</p></td><td char="." align="char"><p>0.667<sup>*</sup></p></td><td char="." align="char"><p>0.526</p></td><td char="." align="char"><p>0.808</p></td><td align="left"><p>2443.262<sup>*</sup></p></td><td char="." align="char"><p>97.913</p></td><td char="." align="char"><p>0.231</p></td></tr><tr><td align="left"><p><underline>Cultural values</underline></p></td><td char="." align="char"><p>11.366<sup>*</sup></p></td><td align="left" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td align="left" /><td char="." align="char" /><td char="." align="char" /></tr><tr><td align="left"><p>Collectivism</p></td><td char="." align="char" /><td align="left"><p>26</p></td><td char="." align="char"><p>0.126</p></td><td char="." align="char"><p>1.007<sup>*</sup></p></td><td char="." align="char"><p>0.760</p></td><td char="." align="char"><p>1.253</p></td><td align="left"><p>197.479</p></td><td char="." align="char"><p>87.340</p></td><td char="." align="char"><p>0.325</p></td></tr><tr><td align="left"><p>Individualism</p></td><td char="." align="char" /><td align="left"><p>53</p></td><td char="." align="char"><p>0.067</p></td><td char="." align="char"><p>0.526<sup>*</sup></p></td><td char="." align="char"><p>0.394</p></td><td char="." align="char"><p>0.657</p></td><td align="left"><p>2692.186</p></td><td char="." align="char"><p>98.068</p></td><td char="." align="char"><p>0.215</p></td></tr><tr><td align="left"><p><underline>Academic subject</underline></p></td><td char="." align="char"><p>99.575<sup>*</sup></p></td><td align="left" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td align="left" /><td char="." align="char" /><td char="." align="char" /></tr><tr><td align="left"><p> Engineering</p></td><td char="." align="char" /><td align="left"><p>7</p></td><td char="." align="char"><p>0.041</p></td><td char="." align="char"><p>0.203<sup>*</sup></p></td><td char="." align="char"><p>0.122</p></td><td char="." align="char"><p>0.285</p></td><td align="left"><p>23.170<sup>*</sup></p></td><td char="." align="char"><p>74.104</p></td><td char="." align="char"><p>0.008</p></td></tr><tr><td align="left"><p>Math</p></td><td char="." align="char" /><td align="left"><p>19</p></td><td char="." align="char"><p>0.076</p></td><td char="." align="char"><p>0.240<sup>*</sup></p></td><td char="." align="char"><p>0.091</p></td><td char="." align="char"><p>0.389</p></td><td align="left"><p>181.750<sup>*</sup></p></td><td char="." align="char"><p>90.096</p></td><td char="." align="char"><p>0.079</p></td></tr><tr><td align="left"><p>Science</p></td><td char="." align="char" /><td align="left"><p>46</p></td><td char="." align="char"><p>0.084</p></td><td char="." align="char"><p>0.843<sup>*</sup></p></td><td char="." align="char"><p>0.679</p></td><td char="." align="char"><p>1.008</p></td><td align="left"><p>1974.328<sup>*</sup></p></td><td char="." align="char"><p>97.721</p></td><td char="." align="char"><p>0.274</p></td></tr><tr><td align="left"><p>Science & Math</p></td><td char="." align="char" /><td align="left"><p>1</p></td><td char="." align="char"><p>0.067</p></td><td char="." align="char"><p>0.802<sup>*</sup></p></td><td char="." align="char"><p>0.671</p></td><td char="." align="char"><p>0.933</p></td><td align="left"><p>0.000</p></td><td char="." align="char"><p>0.000</p></td><td char="." align="char"><p>0.000</p></td></tr><tr><td align="left"><p>STEM</p></td><td char="." align="char" /><td align="left"><p>6</p></td><td char="." align="char"><p>0.236</p></td><td char="." align="char"><p>1.107 <sup>*</sup></p></td><td char="." align="char"><p>0.645</p></td><td char="." align="char"><p>1.570</p></td><td align="left"><p>78.053<sup>*</sup></p></td><td char="." align="char"><p>93.594</p></td><td char="." align="char"><p>0.300</p></td></tr><tr><td align="left"><p><underline>Achievement type</underline></p></td><td char="." align="char"><p>28.395<sup>*</sup></p></td><td align="left" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td align="left" /><td char="." align="char" /><td char="." align="char" /></tr><tr><td align="left"><p>Standardized</p></td><td char="." align="char" /><td align="left"><p>16</p></td><td char="." align="char"><p>0.076</p></td><td char="." align="char"><p>0.229<sup>*</sup></p></td><td char="." align="char"><p>0.080</p></td><td char="." align="char"><p>0.379</p></td><td align="left"><p>299.745<sup>***</sup></p></td><td char="." align="char"><p>94.996</p></td><td char="." align="char"><p>0.073</p></td></tr><tr><td align="left"><p>Non-standardized</p></td><td char="." align="char" /><td align="left"><p>63</p></td><td char="." align="char"><p>0.071</p></td><td char="." align="char"><p>0.784<sup>*</sup></p></td><td char="." align="char"><p>0.645</p></td><td char="." align="char"><p>0.923</p></td><td align="left"><p>2319.761<sup>***</sup></p></td><td char="." align="char"><p>97.327</p></td><td char="." align="char"><p>0.271</p></td></tr><tr><td align="left"><p><underline>Study Design</underline></p></td><td char="." align="char"><p> 22.755*</p></td><td align="left" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td align="left" /><td char="." align="char" /><td char="." align="char" /></tr><tr><td align="left"><p>Pre-post experimental and control</p></td><td char="." align="char" /><td align="left"><p>49</p></td><td char="." align="char"><p>0.073</p></td><td char="." align="char"><p>0.795*</p></td><td char="." align="char"><p>0.651</p></td><td char="." align="char"><p>0.938</p></td><td align="left"><p>670.865</p></td><td char="." align="char"><p>92.845</p></td><td char="." align="char"><p>0.210</p></td></tr><tr><td align="left"><p>Control and experimental group's posts</p></td><td char="." align="char" /><td align="left"><p>13</p></td><td char="." align="char"><p>0.105</p></td><td char="." align="char"><p>0.185*</p></td><td char="." align="char"><p>−0.021</p></td><td char="." align="char"><p>0.391</p></td><td align="left"><p>233.120</p></td><td char="." align="char"><p>94.852</p></td><td char="." align="char"><p>0.119</p></td></tr><tr><td align="left"><p>Experimental group's pre and post</p></td><td char="." align="char" /><td align="left"><p>17</p></td><td char="." align="char"><p>0.122</p></td><td char="." align="char"><p>0.648*</p></td><td char="." align="char"><p>0.409</p></td><td char="." align="char"><p>0.887</p></td><td align="left"><p>1703.600</p></td><td char="." align="char"><p>99.061</p></td><td char="." align="char"><p>0.234</p></td></tr><tr><td align="left"><p><underline>Intervention duration</underline></p></td><td char="." align="char"><p>14.224<sup>*</sup></p></td><td align="left" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td align="left" /><td char="." align="char" /><td char="." align="char" /></tr><tr><td align="left"><p>Less than 4 weeks</p></td><td char="." align="char" /><td align="left"><p>6</p></td><td char="." align="char"><p>0.173</p></td><td char="." align="char"><p>1.047<sup>*</sup></p></td><td char="." align="char"><p>0.708</p></td><td char="." align="char"><p>1.387</p></td><td align="left"><p>44.672<sup>*</sup></p></td><td char="." align="char"><p>93.284</p></td><td char="." align="char"><p>0.295</p></td></tr><tr><td align="left"><p>4–8 weeks</p></td><td char="." align="char" /><td align="left"><p>19</p></td><td char="." align="char"><p>0.190</p></td><td char="." align="char"><p>1.029<sup>*</sup></p></td><td char="." align="char"><p>0.656</p></td><td char="." align="char"><p>1.402</p></td><td align="left"><p>173.251<sup>*</sup></p></td><td char="." align="char"><p>89.610</p></td><td char="." align="char"><p>0.594</p></td></tr><tr><td align="left"><p>Above 8 weeks</p></td><td char="." align="char" /><td align="left"><p>33</p></td><td char="." align="char"><p>0.084</p></td><td char="." align="char"><p>0.461<sup>*</sup></p></td><td char="." align="char"><p>0.296</p></td><td char="." align="char"><p>0.626</p></td><td align="left"><p>2341.134<sup>*</sup></p></td><td char="." align="char"><p>98.634</p></td><td char="." align="char"><p>0.221</p></td></tr><tr><td align="left"><p><underline>Grade level</underline></p></td><td char="." align="char"><p>2.990</p></td><td align="left" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td align="left" /><td char="." align="char" /><td char="." align="char" /></tr><tr><td align="left"><p>Elementary</p></td><td char="." align="char" /><td align="left"><p>29</p></td><td char="." align="char"><p>0.118</p></td><td char="." align="char"><p>0.817<sup>*</sup></p></td><td char="." align="char"><p>0.586</p></td><td char="." align="char"><p>1.048</p></td><td align="left"><p>821.832</p></td><td char="." align="char"><p>96.593</p></td><td char="." align="char"><p>0.354</p></td></tr><tr><td align="left"><p>Middle school</p></td><td char="." align="char" /><td align="left"><p>23</p></td><td char="." align="char"><p>0.091</p></td><td char="." align="char"><p>0.600<sup>*</sup></p></td><td char="." align="char"><p>0.421</p></td><td char="." align="char"><p>0.779</p></td><td align="left"><p>767.405</p></td><td char="." align="char"><p>97.133</p></td><td char="." align="char"><p>0.160</p></td></tr><tr><td align="left"><p>High school</p></td><td char="." align="char" /><td align="left"><p>27</p></td><td char="." align="char"><p>0.561</p></td><td char="." align="char"><p>0.561<sup>*</sup></p></td><td char="." align="char"><p>0.351</p></td><td char="." align="char"><p>0.770</p></td><td align="left"><p>1321130</p></td><td char="." align="char"><p>98.032</p></td><td char="." align="char"><p>0.267</p></td></tr></tbody></table> </ephtml> </p> <p> <emph>Note</emph>. <sups>*</sups><emph>p</emph> <.05</p> <hd id="AN0188452166-23">Learning Activities</hd> <p>The STEM integration<bold>-</bold>achievement link differed across learning activities (<emph>Q</emph><subs>between</subs> = 21.347, <emph>df</emph> = 3, <emph>p</emph> < 0.001, see Table 3). Specifically, inquiry-based learning produced the largest effect size (<emph>g</emph> = 1.051, 95% CI [0.754, 1.349], <emph>p</emph> < 0.001), followed closely by problem-based learning (<emph>g</emph> = 0.941, 95% CI [0.561, 1.321], <emph>p</emph> < 0.001). Design-based learning (<emph>g</emph> = 0.442, 95% CI [0.294, 0.589], <emph>p</emph> < 0.001) and project-based learning (<emph>g</emph> = 0.401, 95% CI [0.255, 0.547], <emph>p</emph> < 0.001) both showed much smaller effect sizes. These results support H-3 (students in problem-based or inquiry-based learning activities outperform other students).</p> <hd id="AN0188452166-24">Engineering Design Skills</hd> <p>Whether engineering design skills were included (or not) did not affect the STEM integration-achievement link (<emph>Q</emph><subs>between</subs> = 0.018, <emph>df</emph> = 1, <emph>p</emph> = 0.892, see Table 3), showing no support for H-4.</p> <hd id="AN0188452166-25">Cultural Values</hd> <p>Cultural values also moderated the STEM integration-achievement link (<emph>Q</emph><subs>between</subs> = 11.366, <emph>df</emph> = 1, <emph>p</emph> < 0.01, see Table 3). Specifically, collectivist culture yielded a larger effect size (<emph>g</emph> = 1.007,95% CI [0.760, 1.253], <emph>p</emph> < 0.001) than an individualistic culture (<emph>g</emph> = 0.526, 95% CI [0.394, 0.657], <emph>p</emph> < 0.001), supporting our H-5 that the STEM integration-achievement link is stronger in collectivist cultures than in individualistic ones.</p> <hd id="AN0188452166-26">Academic Subject</hd> <p>Academic subjects moderated the STEM integration-achievement link (<emph>Q</emph><subs>between</subs> = 99.575, <emph>df</emph> = 4, <emph>p</emph> < 0.001, see Table 3). Specifically, STEM subject achievement produced the largest effect size (<emph>g</emph> = 1.107, 95% CI [0.645, 1.570], <emph>p</emph> < 0.001) followed by science achievement (<emph>g</emph> = 0.843, 95% CI [0.679, 1.008], <emph>p</emph> < 0.001), math achievement (<emph>g</emph> = 0.240, 95% CI [0.091, 0.389], <emph>p</emph> < 0.01), and engineering achievement (<emph>g</emph> = 0.203, 95% CI [0.122, 0.285], <emph>p</emph> < 0.001).</p> <hd id="AN0188452166-27">Standardized Test</hd> <p>Test standardization moderated the STEM integration-achievement link (<emph>Q</emph><subs>between</subs> = 28.395, <emph>df</emph> = 1, <emph>p</emph> < 0.001, see Table 3). Specifically, the effect sizes were larger for non-standardized tests (<emph>g</emph> = 0.784, 95% CI [0.645, 0.923], <emph>p</emph> < 0.001) than for standardized tests (<emph>g</emph> = 0.229, 95% CI [0.080, 0.379], <emph>p</emph> = 0.003).</p> <hd id="AN0188452166-28">Study Design</hd> <p>The study design moderated the STEM integration-achievement link (<emph>Q</emph><subs>between</subs> = 22.755, <emph>df</emph> = 2, <emph>p</emph> < 0.001, see Table 3). Specifically, the effect sizes were largest for controlled experiments with both pre- and post-tests (<emph>g</emph> = 0.795, 95% CI [0.651, 0.938], <emph>p</emph> < 0.001), smaller for a single group with pre- and post-tests (<emph>g</emph> = 0.648, 95% CI [0.409, 0.887], <emph>p</emph> < 0.001) and smallest for a controlled experiment with only post-tests (<emph>g</emph> = 0.185, 95% CI [−0.021, 0.391], <emph>p</emph> > 0.05).</p> <hd id="AN0188452166-29">Intervention Duration</hd> <p>Intervention duration moderated the link between STEM integration and achievement (<emph>Q</emph><subs>between</subs> = 14.224, <emph>df</emph> = 2, <emph>p</emph> < 0.001, see Table 3). Specifically, interventions of less than four weeks (<emph>g</emph> = 1.047, 95% CI [0.708, 1.387], <emph>p</emph> < 0.001) or 4–8 weeks (<emph>g</emph> = 1.029, 95% CI [0.656, 1.402], <emph>p</emph> < 0.001) showed larger effect sizes than those exceeding 8 weeks (<emph>g</emph> = 0.461, 95% CI [0.296, 0.626], <emph>p</emph> < 0.001).</p> <hd id="AN0188452166-30">Grade Level</hd> <p>Grade level did not affect the STEM integration-achievement link (<emph>Q</emph><subs>between</subs> = 2.990, <emph>df</emph> = 2, <emph>p</emph> = 0.224).</p> <hd id="AN0188452166-31">Discussion</hd> <p>This meta-analysis of 79 effect sizes in 40 studies showed a positive link between STEM integration and academic achievement. This STEM integration-achievement link differed across STEM integration type, learning activities, cultural values, academic subjects, standardized versus non-standardized tests, intervention duration, and study design.</p> <hd id="AN0188452166-32">STEM Integration and Student Achievement</hd> <p>The positive effect size of STEM integration on student achievement (g = 0.661) supports the view that STEM integrated activities motivate students by addressing real-world problems, making interdisciplinary connections, and aiding their application of multidisciplinary knowledge—thereby helping them learn more than otherwise (Margot & Kettler, [<reflink idref="bib58" id="ref117">58</reflink>]). This result aligns with those of past meta-analyses (e.g., Guzey et al., [<reflink idref="bib29" id="ref118">29</reflink>]; Robinson et al., [<reflink idref="bib84" id="ref119">84</reflink>]). Hence, educators should advocate training teachers (e.g., via professional development) to teach students via STEM integration to help them learn more.</p> <hd id="AN0188452166-33">Moderators</hd> <p></p> <hd id="AN0188452166-34">Degree of STEM Integration</hd> <p>Among the three degrees of STEM integration, context integration had the largest effect on student achievement. These results align with the following two views. First, a familiar, meaningful, authentic problem situation can enhance students' motivation (Nadelson & Seifert, [<reflink idref="bib68" id="ref120">68</reflink>]) and connect familiar, concrete experiences via more neural links in their brains (Li & Wang, [<reflink idref="bib54" id="ref121">54</reflink>]) to help them learn STEM concepts (Bryan et al., [<reflink idref="bib15" id="ref122">15</reflink>]; Moore & Smith, [<reflink idref="bib67" id="ref123">67</reflink>]). Second, applying them to different contexts helps students understand when and how to apply them to unfamiliar problem contexts (<emph>transfer</emph>, Perkins & Salomon, [<reflink idref="bib79" id="ref124">79</reflink>]). Both are consistent with context integration showing the largest STEM integration effect on student achievement.</p> <p>The weaker, positive results of content integration cohere with teachers' limited content knowledge (Brophy et al., [<reflink idref="bib14" id="ref125">14</reflink>]) and self-efficacy (Lewis et al., [<reflink idref="bib49" id="ref126">49</reflink>]), curriculum constraints (Helle et al., [<reflink idref="bib33" id="ref127">33</reflink>]), and time constraints, which hinder effective STEM content integration and hence, student learning (Roehrig et al., [<reflink idref="bib86" id="ref128">86</reflink>]). For instance, many science teachers who use integrated STEM content activities in their classes lack sufficient knowledge and skills about the novel content (e.g., engineering) to effectively help their students learn it (Hamad et al., [<reflink idref="bib32" id="ref129">32</reflink>]).</p> <p>These results suggest future research testing whether pre-service training or professional development can enhance teachers' STEM knowledge and improve the effectiveness of their STEM content integration for their students' learning. Likewise, future studies can also test whether aligning STEM content integration lessons with the curriculum or giving teachers more time can improve their students' learning outcomes. Among degree of STEM integration, tool integration had the smallest, positive effect on student achievement. This result is consistent with both (a) the fewer links across STEM concepts via tool integration compared to context or content integration, and (b) teachers' unfamiliarity with new technological tools that can hinder their teaching and their students' learning. Still, as only one original study examined tool integration (two effect sizes), the tool integration results require cautious interpretation.</p> <hd id="AN0188452166-35">Learning Activities</hd> <p>Among STEM integration learning activities, inquiry-based learning had the largest positive effect on student achievement, followed closely by problem-based learning, with much smaller positive effects for design-based learning and for project-based learning. These results align with the following views. First, teachers can readily manage the simpler problem-based and inquiry-based learning activities, use them more often (Albion, [<reflink idref="bib3" id="ref130">3</reflink>]), and help their student learn from them (Panasan & Nuangchaler, [<reflink idref="bib76" id="ref131">76</reflink>]). By contrast, teachers often lack STEM knowledge (Dare et al., [<reflink idref="bib21" id="ref132">21</reflink>]), experience with project-based or design-based curricula (Frank et al., [<reflink idref="bib25" id="ref133">25</reflink>]; Nadelson et al., [<reflink idref="bib69" id="ref134">69</reflink>]), time, or curriculum flexibility (Helle et al., [<reflink idref="bib33" id="ref135">33</reflink>]). So, they struggle with projects, design activities, and how to help their students learn from them (Lewis et al., [<reflink idref="bib49" id="ref136">49</reflink>]). The largest positive effect of inquiry-based learning also aligns with the common training of science teachers to teach via inquiry-based learning (NRC, [<reflink idref="bib71" id="ref137">71</reflink>]) and their greater experience with it.</p> <hd id="AN0188452166-36">Engineering Design Skills</hd> <p>Whether STEM integration includes engineering design skills or not does not moderate its effect on achievement. This result does not support some researchers' claims that students' STEM learning benefits from engineering design skills (e.g., Li et al., [<reflink idref="bib53" id="ref138">53</reflink>]), despite the NGSS emphasis on integrating engineering design processes with the scientific method (Dare, et al., [<reflink idref="bib21" id="ref139">21</reflink>]; NRC, [<reflink idref="bib72" id="ref140">72</reflink>]). As these teachers often have limited engineering design expertise they might not implement STEM integration lessons with engineering design skills effectively (Nadelson et al., [<reflink idref="bib69" id="ref141">69</reflink>]).</p> <hd id="AN0188452166-37">Cultural Values</hd> <p>Cultural values moderated the STEM integration-achievement link. Effect sizes in collectivist cultures nearly doubled those in individualist cultures (1.007 > 0.526). This result aligns with the view that students pay more attention to their classmates and support group goals more in collectivist cultures than in individualist cultures (Chiu & Chow, [<reflink idref="bib18" id="ref142">18</reflink>]). Such attention and support enhances learning in integrated STEM activities, which are often done in groups. As students in collectivist cultures especially benefit from integrated STEM group activities, teachers in such cultures should especially consider using them. In individualistic cultures, studies can test whether interventions that foster such student processes (attend to classmates, support group goals) can make STEM integration more effective for student learning.</p> <p>Other meta-analyses also show that culture affects learning interventions (e.g., Lei et al., [<reflink idref="bib46" id="ref143">46</reflink>], [<reflink idref="bib47" id="ref144">47</reflink>]; Dong et al., [<reflink idref="bib22" id="ref145">22</reflink>]). For example, mobile learning had a bigger impact on students' science achievement in collectivist countries than in individualistic ones (e.g., Lu et al., [<reflink idref="bib57" id="ref146">57</reflink>]). Our results match these findings. They are consistent with the view that students in collectivistic cultures value group goals more than individual goals, pay closer attention to one another, recognize one another's problems more readily, and help one another learn more.</p> <hd id="AN0188452166-38">Achievement Measure: Academic Subject and Standardized Tests</hd> <p>The positive link between STEM integration and achievement was strongest for STEM, a bit less for science, and much less for math or engineering design. These results align with the views that (a) STEM tests are better suited for measuring STEM learning, and (b) students engage with, like, and learn more from concrete activities than abstract ones (e.g., physics, Dong et al., [<reflink idref="bib22" id="ref147">22</reflink>]). The stronger concreteness results also align with those of a meta-analysis showing that mobile learning worked best for biology, and progressively less for earth and space sciences, chemistry, and physics (Dong et al., [<reflink idref="bib22" id="ref148">22</reflink>]). As only two studies (with six independent samples) assessed engineering design skills, so those results need cautious interpretation.</p> <p>The positive STEM integration-achievement link was larger in studies using non-standardized tests than those using standardized tests. This result coheres with the view that researcher-designed non-standardized tests fit the content of the STEM activities better but have worse psychometric properties (Li & Ma, [<reflink idref="bib51" id="ref149">51</reflink>]), yielding larger effect sizes compared to uniform, standardized tests.</p> <hd id="AN0188452166-39">Study Design</hd> <p>The STEM integration-achievement link was largest in controlled experiments with both pre- and post-tests, smaller for a single group's pre and post-test scores, and far smaller for a controlled experiment with only post-tests. These results suggest that biases in the latter two types of studies affected the effect size, consistent with past studies (e.g., Kirk, [<reflink idref="bib43" id="ref150">43</reflink>]; Torday & Baluška, [<reflink idref="bib104" id="ref151">104</reflink>]). Hence, future studies should use controlled experiments with both pre- and post-tests.</p> <hd id="AN0188452166-40">Intervention Duration</hd> <p>Compared to longer interventions, shorter ones lasting eight weeks or less had much larger effect sizes. This result fits the claim that more external events during longer interventions interfere with them and dilute their impact (e.g., Nahmias et al., [<reflink idref="bib70" id="ref152">70</reflink>]). As shorter interventions are typically more cost-effective and easier to implement, educators should begin with shorter STEM integration interventions. Conversely, educators can consider about how to increase the effectiveness of longer STEM integration interventions. As only five studies had interventions of less than four weeks in our analysis, those results require cautious interpretation.</p> <hd id="AN0188452166-41">Implications</hd> <p>These results suggest three implications. First, students learn more from STEM integration than traditional teaching methods. Hence, any comprehensive theory of learning must include STEM integration. Also, this result suggests that educators help teachers teach via STEM integration to help their students learn more. Given the larger effect sizes of shorter interventions (eight weeks or less), educators can help teachers start with these, as they are likely more cost-effective and easier to implement than longer ones.</p> <p>Second, students learn more via context integration than other types of integration. Thus, educators should use diverse STEM contexts to motivate their students to learn more. As many teachers have limited content knowledge across disciplines, integrating a new context from another STEM discipline is likely easier for them than integrating content from another STEM discipline.</p> <p>Third, inquiry-based learning and problem-based learning have larger effects than project- or design-based learning. As the latter two are newer, teachers might understand them less. Hence, educators seeking to use project- or design-based learning must understand and tackle this issue before implementing these activities. Otherwise, inquiry- or problem-based learning activities might yield better student learning outcomes.</p> <hd id="AN0188452166-42">Limitations and Future Research</hd> <p>This study has five major limitations: conceptual taxonomy, limited reporting of key variables in original studies, single methodology, language, and few original studies. First, we started classifying STEM integration interventions by types of integration and learning activities, so future scholars can build and test more detailed taxonomies for them. Second, many studies did not report attributes of STEM integration, teachers (e.g., teachers' professional development for STEM), students (e.g., gender, socio-economic status [e.g., Li et al., [<reflink idref="bib50" id="ref153">50</reflink>]; Robinson, [<reflink idref="bib83" id="ref154">83</reflink>]]), or intervention procedures (e.g., teacher interactions with students) so we could not test for their moderation effects. Notably, teachers with better training in STEM subjects might help their students learn more. Future studies can include such information, especially for participants from diverse backgrounds or cultures, to inform meta-analyses examining such differences. Third, as meta-analyses can only show coarse patterns, future studies can also use other methods (e.g., ethnographies, micro-genetic analyses of videotapes of students' problem-solving) to detail intricate mechanisms and contextual differences (e.g., within and across cultures). Fourth, we only examined studies published in English. Future meta-analyses with authors fluent in more languages can include studies published in other languages. Fifth, we only have two studies about tool integration, two studies with less than four weeks' intervention, and two studies with engineering achievement. Hence, we should interpret these results cautiously.</p> <hd id="AN0188452166-43">Conclusion</hd> <p>This study meta-analyzed STEM integration's effect on student achievement using 79 effect sizes from 40 studies. The results showed a positive overall effect of STEM integration on student STEM achievement, indicating its value for improving student learning outcomes. Effect sizes varied across degrees of STEM integration, learning activities, academic subjects, standardization of tests, cultural values, study design, and intervention duration. Whether the integration involved engineering design skills or not did not moderate the STEM integration's effect on student achievement. These results help teacher educators, professional development programs, and teachers design better STEM instruction to improve student learning outcomes.</p> <hd id="AN0188452166-44">Funding</hd> <p>This research was supported by the 2023 Humanities and Social Science Program sponsored by the Ministry of Education of the People's Republic of China (23YJC880019).</p> <hd id="AN0188452166-45">Data Availability</hd> <p>The data used in this research is presented in Table 1 in the Table file. Inquiries about the specific data used in this study can be directed to the corresponding author.</p> <hd id="AN0188452166-46">Declarations</hd> <p></p> <hd id="AN0188452166-47">Ethical Approval</hd> <p>It is not applicable to this study.</p> <hd id="AN0188452166-48">Consent and Participate</hd> <p>It is not applicable to this study.</p> <hd id="AN0188452166-49">Conflicts of Interest</hd> <p>The authors declare that we have no conflict of interest.</p> <hd id="AN0188452166-50">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0188452166-51"> <title> References </title> <blist> <bibl id="bib1" idref="ref8" type="bt">1</bibl> <bibtext> *Acar, D, Tertemiz, N, & Taşdemir, A. (2018). The effects of STEM training on the academic achievement of 4th graders in science and mathematics and their views on STEM training. International Electronic Journal of Elementary Education, 10(4), 505–513. https://doi.org/10.26822/iejee.2018438141</bibtext> </blist> <blist> <bibl id="bib2" type="bt">2</bibl> <bibtext> *Akkaya, M. M, & Benzer, S. (2020). The effect of STEM practices on academic achievement and attitudes of sixth grade students. MOJES: Malaysian Online Journal of Educational Sciences, 8(2), 36–47.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref74" type="bt">3</bibl> <bibtext> Albion PHenderson M, Henderson MJ, Romeo G. Project-, problem-, and inquiry-based learning. Teaching and digital technologies: Big issues and critical questions. 2015; Cambridge University Press: 240-252. 10.1017/CBO9781316091968.024</bibtext> </blist> <blist> <bibl id="bib4" type="bt">4</bibl> <bibtext> *Alameh, S. H. (2018). Effect of science and engineering practices in biology on students attitudes, achievement and engineering design skills [Doctoral dissertation, American University of Beirut]. ProQuest Dissertations Publishing.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref80" type="bt">5</bibl> <bibtext> Ali, M, & Tse, W. C. (2023). Research trends and issues of engineering design process for STEM education in K-12: A bibliometric analysis. International Journal of Education in Mathematics, Science, and Technology (IJEMST), 11(3), 695–727. https://doi.org/10.46328/ijemst.2794</bibtext> </blist> <blist> <bibl id="bib6" idref="ref4" type="bt">6</bibl> <bibtext> Aminger, W, Hough, S, Roberts, S. A, Meier, V, Spina, A. D, Pajela, H, ..., & Bianchini, J. A. (2021). Preservice secondary science teachers' implementation of an NGSS practice: Using mathematics and computational thinking. Journal of Science Teacher Education,32(2), 188–209. https://doi.org/10.1080/1046560X.2020.1805200</bibtext> </blist> <blist> <bibl id="bib7" type="bt">7</bibl> <bibtext> *Anwar, S, Menekse, M, Guzey, S, & Bryan, L. A. (2022). The effectiveness of an integrated STEM curriculum unit on middle school students' life science learning. Journal of Research in Science Teaching, 59(7), 1204-1234. https://doi.org/10.1002/tea.21756</bibtext> </blist> <blist> <bibl id="bib8" idref="ref108" type="bt">8</bibl> <bibtext> *Angwal, Y. A, Saat, R. M, & Sathasivam, R. V. (2019). Preparation and validation of an integrated STEM instructional material for genetic instruction among year 11 science students. Malaysian Online Journal of Educational Sciences, 7(2), 41–56.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref42" type="bt">9</bibl> <bibtext> Bajzek TJ. Thermocouples: A sensor for measuring temperature. IEEE Instrumentation & Measurement Magazine. 2005; 8; 1: 35-40. 10.1109/MIM.2005.1405922</bibtext> </blist> <blist> <bibtext> Barrett BS, Moran AL, Woods JE. Meteorology meets engineering: An interdisciplinary STEM module for middle and early secondary school students. International Journal of STEM Education. 2014; 1; 1: 1-6. 10.1186/2196-7822-1-6</bibtext> </blist> <blist> <bibtext> Becker KH, Park K. Integrative approaches among science, technology, engineering, and mathematics (STEM) subjects on students' learning: A Meta-Analysis. Journal of STEM Education: Innovations and Research. 2011; 12; 5–6: 23-37</bibtext> </blist> <blist> <bibtext> Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods. 2010; 1; 2: 97-111. 10.1002/jrsm.12</bibtext> </blist> <blist> <bibtext> Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to meta-analysis. 2021; John Wiley & Sons. 10.1002/9781119558378</bibtext> </blist> <blist> <bibtext> Brophy S, Klein S, Portsmore M, Rogers C. Advancing engineering education in P-12 classroom. Journal of Engineering Education. 2008; 97; 3: 369-387. 10.1002/j.2168-9830.2008.tb00985.x</bibtext> </blist> <blist> <bibtext> Bryan, L. A, Moore, T. J, Johnson, C. C, & Roehrig, G. H. (2016). Integrated STEM education. In C. C. Johnson, E. E. Peters-Burton, & T. J. Moore (Eds.), STEM road map: A framework for integrated STEM education (pp. 23–37). Routledge.</bibtext> </blist> <blist> <bibtext> Bybee RW. Advancing STEM education: A 2020 vision. Technology and Engineering Teacher. 2010; 70; 1: 30-35</bibtext> </blist> <blist> <bibtext> *Capobianco, B. M, Radloff, J, & Lehman, J. D. (2021). Elementary science teachers' sense-making with learning to implement engineering design and its impact on students' science achievement. Journal of Science Teacher Education, 32(1), 39–61. https://doi.org/10.1080/1046560X.2020.1789267</bibtext> </blist> <blist> <bibtext> Chiu, M. M, & Chow, B. W. Y. (2015). International comparisons of student achievement. In Nata, R. V. (Eds.), Progress in Education (pp. 93–108). Nova Science Publishers, Inc.</bibtext> </blist> <blist> <bibtext> Cobern WW. Constructivism. Journal of Educational and Psychological Consultation. 1993; 4; 1: 105-112. 10.1207/s1532768xjepc0401_8</bibtext> </blist> <blist> <bibtext> *Cotabish, A, Dailey, D, Robinson, A, & Hughes, G. (2013). The effects of a STEM intervention on elementary students' science knowledge and skills. School Science and Mathematics, 113(5), 215–226. https://doi.org/10.1111/ssm.12023</bibtext> </blist> <blist> <bibtext> Dare, E. A, Ellis, J. A, & Roehrig, G. H. (2018). Understanding science teachers' implementations of integrated STEM curricular units through a phenomenological multiple case study. International Journal of STEM Education, 5(4). https://doi.org/10.1186/s40594-018-0101-z.</bibtext> </blist> <blist> <bibtext> Dong, Z, Chiu, M. M, Zhou, S, & Zhang, Z. (2024). The effect of mobile learning on school-aged students' science achievement: A meta-analysis. Education and Information Technologies, 29(1), 517–544. https://doi.org/10.1007/s10639-023-12240-3</bibtext> </blist> <blist> <bibtext> English LD, King D, Smeed J. Advancing integrated STEM learning through engineering design. The Journal of Educational Research. 2017; 110; 3: 255-271. 10.1080/00220671.2016.1264053</bibtext> </blist> <blist> <bibtext> *Fan, S.-C, & Yu, K.-C. (2017). How an integrative STEM curriculum can benefit students in engineering design practices. International Journal of Technology and Design Education, 27(1), 107–129. https://doi.org/10.1007/s10798-015-9328-x</bibtext> </blist> <blist> <bibtext> Frank M, Lavy I, Elata D. Implementing the project-based learning approach in an academic engineering course. International Journal of Technology and Design Education. 2003; 13; 3: 273-288. 10.1023/A:1026192113732</bibtext> </blist> <blist> <bibtext> *Gazibeyoglu, T, & Aydin, A. (2019). The effect of STEM-based activities on 7th grade students' academic achievement in force and energy unit and students' opinions about these activities. Universal Journal of Educational Research, 7(5), 1275–1285. https://doi.org/10.13189/ujer.2019.070513</bibtext> </blist> <blist> <bibtext> Gentile L, Caudill L, Fetea M, Hill A, Hoke K, Lawson B, Szajda D. Challenging disciplinary boundaries in the first year: A new introductory integrated science course for STEM majors. Journal of College Science Teaching. 2012; 41; 5: 44-50</bibtext> </blist> <blist> <bibtext> Guzey SS, Moore TJ, Harwell M, Moreno M. STEM integration in middle school life science. Journal of Science Education and Technology. 2016; 25; 4: 550-560. 10.1007/s10956-016-9612-x</bibtext> </blist> <blist> <bibtext> *Guzey, S. S, Harwell, M, Moreno, M, Peralta, Y, & Moore, T. J. (2017). The impact of design-based STEM integration curricula on student achievement in engineering, science, and mathematics. Journal of Science Education and Technology, 26(2), 207-222. https://doi.org/10.1007/s10956-016-9673-x</bibtext> </blist> <blist> <bibtext> Guzey SS, Caskurlu S, Kozan KJohnson C, Mohr-Schroeder M, Moore T, English L. Integrated STEM pedagogies and student learning. Handbook of research on STEM education. 2020; Routledge: 65-75. 10.4324/9780429021381-8</bibtext> </blist> <blist> <bibtext> *Hasançebi, F, Güner, Ö, Kutru, C, & Hasancebi, M. (2021). Impact of Stem integrated argumentation-based inquiry applications on students' academic success, reflective thinking and creative thinking skills. Participatory Educational Research, 8(4), 274–296. https://doi.org/10.17275/per.21.90.8.4</bibtext> </blist> <blist> <bibtext> Hamad S, Tairab H, Wardat Y, Lutfieh R, AlArabi K, Yousif M, Abu-Al-Aish A, Stoica G. Understanding science teachers' implementations of integrated STEM. Sustainability. 2022; 14; 6: 3594. 10.3390/su14063594</bibtext> </blist> <blist> <bibtext> Helle L, Tynjälä P, Olkinuora E. Project-based learning in post-secondary education–theory, practice and rubber sling shots. Higher Education. 2006; 51; 2: 287-314. 10.1007/s10734-004-6386-5</bibtext> </blist> <blist> <bibtext> Hess, J. L, & Sorge, B, & Feldhaus, C. (2016). The efficacy of Project Lead the Way. Paper presented at 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. https://doi.org/10.18260/p.26151</bibtext> </blist> <blist> <bibtext> Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring Inconsistency in Meta-Analyses. Bmj. 2003; 327; 7414: 557-560. 10.1136/bmj.327.7414.557</bibtext> </blist> <blist> <bibtext> Hmelo-Silver CE, Duncan RG, Chinn CA. Scaffolding and achievement in problem-based and inquiry learning. Educational Psychologist. 2007; 42; 2: 99-107. 10.1080/0046152070126336</bibtext> </blist> <blist> <bibtext> *Hsiao, H.-S, Yu, K.-C, Chang, Y.-S, Chien, Y.-H, Lin, K.-Y, Lin, C.-Y, Chen, J.-C, Chen, J.-H, & Lin, Y.-W. (2017). The study on integrating the design thinking model and STEM activity unit for senior high school living technology course. 2017 7th World Engineering Education Forum (WEEF), 383–390. https://doi.org/10.1109/WEEF.2017.8467111</bibtext> </blist> <blist> <bibtext> *Izgi, S. & Kalayci, S. (2020). The effect of the STEM approach based on the 5E model on academic achievement and scientific process skills: the transformation of electrical energy. International Journal of Education Technology and Scientific Researches. 5(13), 1578–1629. Retrieved Nov 18, 2024, from https://<ulink href="http://www.ijetsar.com/Makaleler/780384334%5f11.%201578-1629%20serpil%20kalayc%c4%b1.pdf">www.ijetsar.com/Makaleler/780384334%5f11.%201578-1629%20serpil%20kalayc%c4%b1.pdf</ulink></bibtext> </blist> <blist> <bibtext> *Jahan, A. (2018). Impact of project Lead the Way™ engineering program on student achievement (Publication No. 10931041) [Doctoral dissertation, Lamar University-Beaumont]. ProQuest Dissertations Publishing.</bibtext> </blist> <blist> <bibtext> *James, J. S. (2014). Science, technology, engineering, and mathematics (STEM) curriculum and seventh grade mathematics and science achievement (Publication No. 3614935) [Doctoral dissertation, Grand Canyon University]. ProQuest Dissertations Publishing.</bibtext> </blist> <blist> <bibtext> *Kağnıcı, A, & Sadi, Ö. (2021). Students' conceptions of learning biology and achievement after STEM activity–enriched instruction. IE: Inquiry in Education, 13(1), Article 7. https://digitalcommons.nl.edu/ie/vol13/iss1/7</bibtext> </blist> <blist> <bibtext> *Kelley, T. R, Sung, E, Han, J, & Knowles, J. G. (2023). Impacting secondary students' STEM knowledge through collaborative STEM teacher partnerships. International Journal of Technology and Design Education, 33(4), 1563-1584. https://doi.org/10.1007/s10798-022-09783-w</bibtext> </blist> <blist> <bibtext> Kirk REMillsap RE, Maydeu-Olivares A. Experimental design. The sage handbook of quantitative methods in psychology. 2009; Sage Publications: 23-45. 10.4135/9780857020994.n2</bibtext> </blist> <blist> <bibtext> Kokotsaki D, Menzies V, Wiggins A. Project-based learning. Improving Schools. 2016; 19; 3: 267-277. 10.1177/1365480216659733</bibtext> </blist> <blist> <bibtext> *Kurt, M, & Benzer, S. (2020). An Investigation on the effect of STEM practices on sixth grade students' academic achievement, problem solving skills, and attitudes towards STEM. Journal of Science Learning, 3(2), 79–88. https://doi.org/10.17509/jsl.v3i2.21419</bibtext> </blist> <blist> <bibtext> Lei H, Wang C, Chiu MM, Chen S. Do educational games affect students' achievement emotions? Evidence from a meta-analysis. Journal of Computer Assisted Learning. 2022; 38; 4: 946-959. 10.1111/jcal.12664</bibtext> </blist> <blist> <bibtext> Lei H, Chiu MM, Wang D, Wang C, Xie T. Effects of Game-Based Learning on Students' Achievement in Science: A Meta-Analysis. Journal of Educational Computing Research. 2022; 60; 6: 1373-1398. 10.1177/07356331211064543</bibtext> </blist> <blist> <bibtext> LeTendre GK, Hofer BK, Shimizu H. What is tracking?. American Educational Research Journal. 2003; 40: 43-89. 10.3102/00028312040001043</bibtext> </blist> <blist> <bibtext> Lewis F, Edmonds J, Fogg-Rogers L. Engineering science education. International Journal of Science Education. 2021; 43; 5: 793-822. 10.1080/09500693.2021.1887544</bibtext> </blist> <blist> <bibtext> *Li, L, Tripathy, R, Salguero, K, & McCarthy, B. (2018). Evaluation of learning by making i3 project: STEM success for rural schools. WestEd. Retrieved Nov 18, 2024, from https://files.eric.ed.gov/fulltext/ED594016.pdf</bibtext> </blist> <blist> <bibtext> Li Q, Ma X. A meta-analysis of the effects of computer technology on school students' mathematics learning. Educational Psychology Review. 2010; 22: 215-243. 10.1007/s10648-010-9125-8</bibtext> </blist> <blist> <bibtext> *Li, Y, Huang, Z, Jiang, M, & Chang, T. W. (2016). The effect on pupils' science performance and problem-solving ability through Lego. Journal of Educational Technology & Society, 19(3), 143–156. https://doi.org/10.2307/jeductechsoci.19.3.143</bibtext> </blist> <blist> <bibtext> Li Y, Schoenfeld AH, diSessa AA, Grasser AC, Benson LC, English LD, Duschl RA. Design and design thinking in STEM education. Journal for STEM Education Research. 2019; 2; 2: 93-104. 10.1007/s41979-019-00020-z</bibtext> </blist> <blist> <bibtext> Li X, Wang W. Exploring spatial cognitive process among STEM students and its role in STEM education: A cognitive neuroscience perspective. Science & Education. 2021; 30; 1: 121-145. 10.1007/s11191-020-00167-x</bibtext> </blist> <blist> <bibtext> *Lie, R, Selcen Guzey, S, & Moore, T. J. (2019). Implementing engineering in diverse upper elementary and middle school science classrooms. Journal of Science Education and Technology, 28(2), 104–117. https://doi.org/10.1007/s10956-018-9751-3</bibtext> </blist> <blist> <bibtext> *Lippert, R. J, & Seals, T. R. (2023). Impacting African American Student Achievement in the Middle School STEM Classroom by Teaching Mathematics Through Arts Integration and Design Thinking (Doctoral dissertation, University of Missouri-Saint Louis). Retrieved Nov 17, 2024, from https://irl.umsl.edu/dissertation/1301</bibtext> </blist> <blist> <bibtext> Lu Z, Chiu MM, Cui Y, Mao W, Lei H. Effects of game-based learning on students' computational thinking: A meta-analysis. Journal of Educational Computing Research. 2023; 61; 1: 235-256. 10.1177/073563312211007</bibtext> </blist> <blist> <bibtext> Margot KC, Kettler T. Teachers' perception of STEM integration and education. International Journal of STEM Education. 2019; 6; 1: 1-16. 10.1186/s40594-018-0151-2</bibtext> </blist> <blist> <bibtext> Massachusetts Department of Education. (2006). Massachusetts science and technology/engineering curriculum framework. Massachusetts Department of Education. Retrieved Nov 18, 2024, from https://files.eric.ed.gov/fulltext/ED508413.pdf</bibtext> </blist> <blist> <bibtext> McAtavey J, Nikolovska I. Team collectivist culture: A remedy for creating team effectiveness. Human Resource Development Quarterly. 2010; 21; 3: 307-316. 10.1002/hrdq.20039</bibtext> </blist> <blist> <bibtext> *McClain, M. L. (2015). The effect of STEM education on mathematics achievement of fourth-grade underrepresented minority students (Publication No. 3723284) [Doctoral dissertation, Capella University]. ProQuest Dissertations Publishing.</bibtext> </blist> <blist> <bibtext> *McGehee, N. J. L. (2015). Project-based learning in STEM and act achievement [Doctoral dissertation, Tennessee Technological University]. Retrieved Nov 17, 2024, from https://<ulink href="http://www.proquest.com/docview/1689659663?pq-origsite=gscholar&fromopenview=true">www.proquest.com/docview/1689659663?pq-origsite=gscholar&fromopenview=true</ulink></bibtext> </blist> <blist> <bibtext> *McHugh, L. (2016). The integration of mathematics in middle school science (Publication No.10140739) [Doctoral dissertation, State University of New York at Stony Brook]. ProQuest Dissertations Publishing.</bibtext> </blist> <blist> <bibtext> Merrill, C. (2001). Integrated technology, mathematics, and science education. Journal of Industrial Teacher Education, 38, 45–61. Retrieved Nov 17, 2024, from https://scholar.lib.vt.edu/ejournals/JITE/v38n3/merrill.html</bibtext> </blist> <blist> <bibtext> Moher, D, Liberati, A, Tetzlaff, J, Altman, D. G, & PRISMA Group*. (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Annals of Internal Medicine, 151(4), 264–269. https://doi.org/10.7326/0003-4819-151-4-200908180-00135</bibtext> </blist> <blist> <bibtext> Moore TJ, Johnston AC, Glancy AWJohnson CC, Mohr-Schroeder MJ, Moore TJ, English LD. STEM integration: A synthesis of conceptual frameworks and definitions. Handbook of research on STEM education. 2020; Routledge: 3-16. 10.4324/9780429021381-2</bibtext> </blist> <blist> <bibtext> Moore, T. J, & Smith, K. A. (2014). Advancing the State of the Art of STEM Integration. Journal of STEM Education, 15(1), 5. Retrieved Nov 17, 2024, from https://karlsmithmn.org/wp-content/uploads/2017/08/Moore-Smith-JSTEMEd-GuestEditorialF.pdf</bibtext> </blist> <blist> <bibtext> Nadelson LS, Seifert AL. Integrated STEM defined: Contexts, challenges, and the future. The Journal of Educational Research. 2017; 110; 3: 221-223. 10.1080/00220671.2017.1289775</bibtext> </blist> <blist> <bibtext> Nadelson, L, Sias, C. M, & Seifert, A. (2016). Challenges for integrating engineering into K-12 curriculum: Indicators of K-12 teachers' propensity to adopt innovation. Paper presented at the ASEE Annual Conference & Exposition, New Orleans, LA. Retrieved Sept 14, 2023, from https://peer.asee.org/challenges-for-integrating-engineering-into-the-k-12-curriculum-indicators-of-k-12-teachers-propensity-to-adopt-innovation</bibtext> </blist> <blist> <bibtext> Nahmias AS, Pellecchia M, Stahmer AC, Mandell DS. Effectiveness of community-based early intervention for children with autism spectrum disorder: A meta-analysis. Journal of Child Psychology and Psychiatry. 2019; 60; 11: 1200-1209. 10.1111/jcpp.13073</bibtext> </blist> <blist> <bibtext> National Research Council (NRC). (1996). National science education standards. National Academies Press.</bibtext> </blist> <blist> <bibtext> National Research Council [NRC]. (2013). Next generation science standards: For states, by states. National Academies Press.</bibtext> </blist> <blist> <bibtext> National Center for Education Statistics (NCES). (2024). What is NAEP? Retrieved Aug 9, 2024, from https://nces.ed.gov/nationsreportcard/</bibtext> </blist> <blist> <bibtext> Organization for Economic Cooperation and Development (OECD). (2024). About PISA. Retrieved from Aug 9, 2024, https://<ulink href="http://www.oecd.org/en/about/programmes/pisa.html">www.oecd.org/en/about/programmes/pisa.html</ulink></bibtext> </blist> <blist> <bibtext> Olivarez, N. (2012). The Impact of a STEM program on academic achievement of eighth grade students in a south Texas middle school (Publication No. 3549798) [Doctoral dissertation, Texas A&M University-Corpus Christi]. ProQuest Dissertations Publishing.</bibtext> </blist> <blist> <bibtext> Panasan M, Nuangchalerm P. Learning outcomes of project-based and inquiry-based learning activities. Online Submission. 2010; 6; 2: 252-255</bibtext> </blist> <blist> <bibtext> *Parlakay, E. S, & Koç, Y. (2020). An investigation of the effectiveness of STEM practices on fifth grade students' academic achievement and motivations at the unit "Exploring and Knowing The World of Living Creatures". International Journal of Progressive Education, 16(1), 125–137. https://doi.org/10.29329/ijpe.2020.228.1</bibtext> </blist> <blist> <bibtext> Pedaste M, Mäeots M, Leijen Ä, Sarapuu S. Improving students' inquiry skills through reflection and self-regulation scaffolds. Technology, Instruction, Cognition and Learning. 2012; 9; 1–2: 81-95</bibtext> </blist> <blist> <bibtext> Perkins DN, Salomon G. Knowledge to go: A Motivational and Dispositional View of Transfer. Educational Psychologist. 2012; 47; 3: 248-258. 10.1080/00461520.2012.693354</bibtext> </blist> <blist> <bibtext> *Pozarski Connolly, C. J. (2017). Evaluating the effectiveness of project recharge [Doctoral dissertation, University of Nevada, Reno]. ProQuest Dissertations Publishing.</bibtext> </blist> <blist> <bibtext> Puente SG, Van Eijck M, Jochems W. Empirical validation of characteristics of design-based learning in higher education. International Journal of Engineering Education. 2013; 29; 2: 491-503</bibtext> </blist> <blist> <bibtext> *Rehmat, A. P, & Hartley, K. (2020). Building engineering awareness. Interdisciplinary Journal of Problem-Based Learning, 14(1). https://doi.org/10.14434/ijpbl.v14i1.28636</bibtext> </blist> <blist> <bibtext> *Robinson, N. (2016). A case study exploring the effects of using an integrative STEM curriculum on eighth grade students? Performance and engagement in the mathematics classroom [Doctoral dissertation, Georgia State University]. ProQuest Dissertations Publishing.</bibtext> </blist> <blist> <bibtext> *Robinson, A, Dailey, D, Hughes, G, & Cotabish, A. (2014). The effects of a science-focused STEM intervention on gifted elementary students' science knowledge and skills. Journal of Advanced Academics,25(3), 189–213. https://doi.org/10.1177/1932202X14533799</bibtext> </blist> <blist> <bibtext> Roehrig GH, Dare EA, Ring-Whalen E, Wieselmann JR. Understanding coherence and integration in integrated STEM curriculum. International Journal of STEM Education. 2021; 8: 1-21. 10.1186/s40594-020-00259-8</bibtext> </blist> <blist> <bibtext> Roehrig, G. H, Moore, T. J, Wang, H. H, & Park, M. S. (2012). Is adding the E enough? Investigating the impact of K‐12 engineering standards on the implementation of STEM integration. School Science and Mathematics, 112(1), 31–44. https://doi.org/10.1111/j.1949-8594.2011.00112.x</bibtext> </blist> <blist> <bibtext> Ross JM, Peterman K, Daugherty JL, Custer RL. An engineering innovation tool: providing science educators a picture of engineering in their classroom. Journal of STEM Education: Innovations & Research. 2018; 19; 2: 13-18</bibtext> </blist> <blist> <bibtext> Rothstein HR, Sutton AJ, Borenstein MRothstein HR, Sutton AJ, Borenstein M. Publication bias in meta-analysis: A brief overview. Publication bias in meta-analysis: Prevention, assessment, and adjustments. 2005; Wiley: 1-10. 10.1002/0470870168</bibtext> </blist> <blist> <bibtext> *Sabag, N, & Trotskovsky, E. (2013). Using lab experiments in electric circuits to promote achievements in mathematics. 2013 IEEE Global Engineering Education Conference (EDUCON), 123–129. https://doi.org/10.1109/EduCon.2013.6530096</bibtext> </blist> <blist> <bibtext> Saraç H. The Effect of Science, Technology, Engineering and Mathematics-STEM Educational Practices on Students' Learning Outcomes: A Meta-Analysis Study. Turkish Online Journal of Educational Technology-TOJET. 2018; 17; 2: 125-142</bibtext> </blist> <blist> <bibtext> *Sarican, G, & Akgunduz, D. (2018). The impact of integrated STEM education on academic achievement, reflective thinking skills towards problem solving and permanence in learning in science education. Cypriot Journal of Educational Sciences, 13(1), 94-113.</bibtext> </blist> <blist> <bibtext> *Sauder, L. D. (2023). Integrated STEM Learning Activity: Effect on Student Engagement and Learning (Publication No. 30489534) (Doctoral dissertation, University of South Dakota). ProQuest Dissertations Publishing.</bibtext> </blist> <blist> <bibtext> Servant-Miklos, V. (2020). Problem-oriented project work and problem-based learning: "Mind the Gap!". Interdisciplinary Journal of Problem-Based Learning, 14(1). https://doi.org/10.14434/ijpbl.v14i1.28596</bibtext> </blist> <blist> <bibtext> *Shelden, T. C. (2021). Making and Math: Exploring How Integrated Stem Maker Units Impact Upper Elementary Math Development [Doctoral dissertation, University of Massachusetts Lowell]. ProQuest Dissertations Publishing.</bibtext> </blist> <blist> <bibtext> Siverling EA, Suazo-Flores E, Mathis CA, Moore TJ. Students' use of STEM content in design justifications during engineering design-based STEM integration. School Science and Mathematics. 2019; 119; 8: 457-474. 10.1111/ssm.12373</bibtext> </blist> <blist> <bibtext> Siregar, N. C, Rosli, R, Maat, S. M, & Capraro, M. M. (2019). The effect of science, technology, engineering and mathematics (STEM) program on students' achievement in mathematics: A meta-analysis. International Electronic Journal of Mathematics Education, 15(1), em0549.</bibtext> </blist> <blist> <bibtext> *Srihongsa, C, Santiboon, T, & Ponkham, K. (2017). Assessing students' critical thinking abilities and science attitudes for enhancing their achievements through the instructional approaching management with the STEM education instructional method of secondary students at the 10th grade level. European Journal of Education Studies, 3(5),377-404. https://doi.org/10.5281/ZENODO.546597</bibtext> </blist> <blist> <bibtext> *Stitham, R. (2018). The Effects of an Elementary STEM Intervention on Fourth-Grade Student Outcomes in Language Arts and Math (Publication No.13419109) [Doctoral dissertation, Concordia University (Oregon)]. ProQuest Dissertations Publishing.</bibtext> </blist> <blist> <bibtext> Struyf A, De Loof H, Boeve-de Pauw J, Van Petegem P. Students' engagement in different STEM learning environments: Integrated STEM education as promising practice?. International Journal of Science Education. 2019; 41; 10: 1387-1407. 10.1080/09500693.2019.1607983</bibtext> </blist> <blist> <bibtext> Sung YT, Chang KE, Liu TC. The effects of integrating mobile devices with teaching and learning on students' learning performance. Computers & Education. 2016; 94: 252-275. 10.1016/j.compedu.2015.11.008</bibtext> </blist> <blist> <bibtext> *Sunyoung, H. A. N, Rosli, R, Capraro, M. M, & Capraro, R. M. (2016). The effect of science, technology, engineering and mathematics (STEM) project based learning (PBL) on students' achievement in four mathematics topics. Journal of Turkish Science Education, 13(special), 3–29.</bibtext> </blist> <blist> <bibtext> *Tati, T, Firman, H, & Riandi, R. I. O. P. (2017). The effect of STEM learning through the project of designing boat model toward student STEM literacy. Journal of Physics: Conference Series, 895(1), 012157. https://doi.org/10.1088/17426596/895/1/012157</bibtext> </blist> <blist> <bibtext> Tian L, Li Y, Li PP, Bodla AA. Leader–member skill distance, team cooperation, and team performance: A cross-culture study in the context of sport teams. International Journal of Intercultural Relations. 2015; 49: 183-197. 10.1016/j.ijintrel.2015.10.005</bibtext> </blist> <blist> <bibtext> Torday, J. S, & Baluška, F. (2019). Why control an experiment?: From empiricism, via consciousness, toward Implicate Order. EMBO Reports,20(10), e49110. https://doi.org/10.1080/00461520.2012.693354</bibtext> </blist> <blist> <bibtext> *Tsai, L. T, Chang, C. C, & Cheng, H. T. (2021). Effect of a STEM-oriented course on students' marine science motivation, interest, and achievements. Journal of Baltic Science Education, 20(1), 134–145. https://doi.org/10.33225/jbse/21.20.134</bibtext> </blist> <blist> <bibtext> *Vallera, F. L, & Bodzin, A. M. (2020). Integrating STEM with AgLIT (agricultural literacy through innovative technology): The efficacy of a project-based curriculum for upper-primary students. International Journal of Science and Mathematics Education, 18(3), 419-439. https://doi.org/10.1007/s10763-019-09979-y</bibtext> </blist> <blist> <bibtext> Vasquez, J. A, Cary, S, & Comer, M. (2013). STEM lesson essentials, grades 3–8: Integrating science, technology, engineering, and mathematics. Heinemann.</bibtext> </blist> <blist> <bibtext> Vasquez JA. STEM–Beyond the Acronym. Educational Leadership. 2015; 72; 4: 10-15</bibtext> </blist> <blist> <bibtext> Wahono B, Lin PL, Chang CY. Evidence of STEM enactment effectiveness in Asian student learning outcomes. International Journal of STEM Education. 2020; 7; 1: 1-18. 10.1186/s40594-020-00236-1</bibtext> </blist> <blist> <bibtext> Wang HH, Knobloch NA. Levels of STEM integration through agriculture, food, and natural resources. Journal of Agricultural Education. 2018; 59; 3: 258-277. 10.5032/jae.2018.03258</bibtext> </blist> <blist> <bibtext> Wang, H. H, & Knobloch, N. A. (2022). Preservice educators' interpretations and pedagogical benefits of a STEM integration through agriculture, food and natural resources rubric. Journal of Pedagogical Research, 6(2), 4–28. https://doi.org/10.33902/JPR.202213513</bibtext> </blist> <blist> <bibtext> Williams, E. V. (2020). Investigating the impact of the integrated STEM program on student test scores in Jamaica [Doctoral dissertation, William Howard Taft University]. ProQuest Dissertations Publishing.</bibtext> </blist> <blist> <bibtext> Wong HLH. Critique: A communicative event in design education. Visible Language. 2011; 45; 3: 221-247</bibtext> </blist> <blist> <bibtext> Wu Y, Zhao B, Wei B, Li Y. Cultural or economic factors? Which matters more for collaborative problem-solving skills: Evidence from 31 countries. Personality and Individual Differences. 2022; 190: 111497. 10.1016/j.paid.2021.111497</bibtext> </blist> <blist> <bibtext> *Yaki, A. A, Saat, R. M, Sathasivam, R. V, & Zulnaidi, H. (2019). Enhancing science achievement utilising an integrated STEM approach. Malaysian Journal of Learning and Instruction, 16(1), 181-205.</bibtext> </blist> <blist> <bibtext> Yew EHJ, Goh K. Problem-based learning: An overview of its process and impact on learning. Health Professions Education. 2016; 2; 2: 75-79. 10.1016/j.hpe.2016.01.004</bibtext> </blist> <blist> <bibtext> You HS. Why teach science with an interdisciplinary approach: History, trends, and conceptual frameworks. Journal of Education and Learning. 2017; 6; 4: 66. 10.5539/jel.v6n4p66</bibtext> </blist> <blist> <bibtext> Yücelyiğit S, Toker Z. A meta-analysis on STEM studies in early childhood education. Turkish Journal of Education. 2021; 10; 1: 23-36. 10.19128/turje.783724</bibtext> </blist> <blist> <bibtext> Zeng, S.B. & Yao, J.J. (2020). Identifying the "Best Evidence": How to use meta-analysis to conduct a literature review—A case of STEM education's effect on students' academic achievement]. Journal of East China Normal University (Educational Sciences Edition), 38(6), 70–85. https://doi.org/10.16382/j.cnki.1000-5560.2020.06.005</bibtext> </blist> </ref> <aug> <p>By Shuqi Zhou; Zehua Dong; Hui Hui Wang and Ming Ming Chiu</p> <p>Reported by Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib67" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib99" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib28" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib23" firstref="ref5"></nolink> <nolink nlid="nl5" bibid="bib84" firstref="ref6"></nolink> <nolink nlid="nl6" bibid="bib72" firstref="ref7"></nolink> <nolink nlid="nl7" bibid="bib64" firstref="ref9"></nolink> <nolink nlid="nl8" bibid="bib62" firstref="ref10"></nolink> <nolink nlid="nl9" bibid="bib66" firstref="ref11"></nolink> <nolink nlid="nl10" bibid="bib85" firstref="ref12"></nolink> <nolink nlid="nl11" bibid="bib15" firstref="ref13"></nolink> <nolink nlid="nl12" bibid="bib11" firstref="ref14"></nolink> <nolink nlid="nl13" bibid="bib118" firstref="ref15"></nolink> <nolink nlid="nl14" bibid="bib96" firstref="ref16"></nolink> <nolink nlid="nl15" bibid="bib109" firstref="ref17"></nolink> <nolink nlid="nl16" bibid="bib90" firstref="ref18"></nolink> <nolink nlid="nl17" bibid="bib119" firstref="ref19"></nolink> <nolink nlid="nl18" bibid="bib16" firstref="ref21"></nolink> <nolink nlid="nl19" bibid="bib108" firstref="ref22"></nolink> <nolink nlid="nl20" bibid="bib10" firstref="ref26"></nolink> <nolink nlid="nl21" bibid="bib27" firstref="ref27"></nolink> <nolink nlid="nl22" bibid="bib68" firstref="ref33"></nolink> <nolink nlid="nl23" bibid="bib54" firstref="ref34"></nolink> <nolink nlid="nl24" bibid="bib79" firstref="ref35"></nolink> <nolink nlid="nl25" bibid="bib110" firstref="ref36"></nolink> <nolink nlid="nl26" bibid="bib111" firstref="ref37"></nolink> <nolink nlid="nl27" bibid="bib95" firstref="ref39"></nolink> <nolink nlid="nl28" bibid="bib14" firstref="ref43"></nolink> <nolink nlid="nl29" bibid="bib25" firstref="ref44"></nolink> <nolink nlid="nl30" bibid="bib69" firstref="ref45"></nolink> <nolink nlid="nl31" bibid="bib32" firstref="ref46"></nolink> <nolink nlid="nl32" bibid="bib33" firstref="ref47"></nolink> <nolink nlid="nl33" bibid="bib58" firstref="ref51"></nolink> <nolink nlid="nl34" bibid="bib107" firstref="ref54"></nolink> <nolink nlid="nl35" bibid="bib117" firstref="ref55"></nolink> <nolink nlid="nl36" bibid="bib19" firstref="ref57"></nolink> <nolink nlid="nl37" bibid="bib93" firstref="ref58"></nolink> <nolink nlid="nl38" bibid="bib116" firstref="ref59"></nolink> <nolink nlid="nl39" bibid="bib78" firstref="ref60"></nolink> <nolink nlid="nl40" bibid="bib36" firstref="ref62"></nolink> <nolink nlid="nl41" bibid="bib34" firstref="ref64"></nolink> <nolink nlid="nl42" bibid="bib44" firstref="ref65"></nolink> <nolink nlid="nl43" bibid="bib81" firstref="ref66"></nolink> <nolink nlid="nl44" bibid="bib59" firstref="ref67"></nolink> <nolink nlid="nl45" bibid="bib21" firstref="ref69"></nolink> <nolink nlid="nl46" bibid="bib49" firstref="ref73"></nolink> <nolink nlid="nl47" bibid="bib76" firstref="ref75"></nolink> <nolink nlid="nl48" bibid="bib30" firstref="ref77"></nolink> <nolink nlid="nl49" bibid="bib87" firstref="ref83"></nolink> <nolink nlid="nl50" bibid="bib48" firstref="ref85"></nolink> <nolink nlid="nl51" bibid="bib18" firstref="ref86"></nolink> <nolink nlid="nl52" bibid="bib60" firstref="ref88"></nolink> <nolink nlid="nl53" bibid="bib103" firstref="ref89"></nolink> <nolink nlid="nl54" bibid="bib113" firstref="ref90"></nolink> <nolink nlid="nl55" bibid="bib114" firstref="ref91"></nolink> <nolink nlid="nl56" bibid="bib24" firstref="ref92"></nolink> <nolink nlid="nl57" bibid="bib22" firstref="ref93"></nolink> <nolink nlid="nl58" bibid="bib73" firstref="ref94"></nolink> <nolink nlid="nl59" bibid="bib74" firstref="ref95"></nolink> <nolink nlid="nl60" bibid="bib51" firstref="ref96"></nolink> <nolink nlid="nl61" bibid="bib37" firstref="ref97"></nolink> <nolink nlid="nl62" bibid="bib80" firstref="ref98"></nolink> <nolink nlid="nl63" bibid="bib17" firstref="ref99"></nolink> <nolink nlid="nl64" bibid="bib43" firstref="ref100"></nolink> <nolink nlid="nl65" bibid="bib104" firstref="ref101"></nolink> <nolink nlid="nl66" bibid="bib70" firstref="ref103"></nolink> <nolink nlid="nl67" bibid="bib65" firstref="ref104"></nolink> <nolink nlid="nl68" bibid="bib75" firstref="ref106"></nolink> <nolink nlid="nl69" bibid="bib112" firstref="ref107"></nolink> <nolink nlid="nl70" bibid="bib13" firstref="ref109"></nolink> <nolink nlid="nl71" bibid="bib100" firstref="ref110"></nolink> <nolink nlid="nl72" bibid="bib12" firstref="ref111"></nolink> <nolink nlid="nl73" bibid="bib35" firstref="ref112"></nolink> <nolink nlid="nl74" bibid="bib88" firstref="ref113"></nolink> <nolink nlid="nl75" bibid="bib40" firstref="ref115"></nolink> <nolink nlid="nl76" bibid="bib437" firstref="ref116"></nolink> <nolink nlid="nl77" bibid="bib29" firstref="ref118"></nolink> <nolink nlid="nl78" bibid="bib86" firstref="ref128"></nolink> <nolink nlid="nl79" bibid="bib71" firstref="ref137"></nolink> <nolink nlid="nl80" bibid="bib53" firstref="ref138"></nolink> <nolink nlid="nl81" bibid="bib46" firstref="ref143"></nolink> <nolink nlid="nl82" bibid="bib47" firstref="ref144"></nolink> <nolink nlid="nl83" bibid="bib57" firstref="ref146"></nolink> <nolink nlid="nl84" bibid="bib50" firstref="ref153"></nolink> <nolink nlid="nl85" bibid="bib83" firstref="ref154"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1489580
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A Meta-Analysis of STEM Integration on Student Academic Achievement
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Shuqi+Zhou%22">Shuqi Zhou</searchLink><br /><searchLink fieldCode="AR" term="%22Zehua+Dong%22">Zehua Dong</searchLink><br /><searchLink fieldCode="AR" term="%22Hui+Hui+Wang%22">Hui Hui Wang</searchLink><br /><searchLink fieldCode="AR" term="%22Ming+Ming+Chiu%22">Ming Ming Chiu</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Research+in+Science+Education%22"><i>Research in Science Education</i></searchLink>. 2025 55(5):1273-1302.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 30
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Information Analyses
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Meta+Analysis%22">Meta Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22STEM+Education%22">STEM Education</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Achievement%22">Academic Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Effect+Size%22">Effect Size</searchLink><br /><searchLink fieldCode="DE" term="%22Problem+Based+Learning%22">Problem Based Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Science+Achievement%22">Science Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Achievement%22">Mathematics Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Engineering+Education%22">Engineering Education</searchLink><br /><searchLink fieldCode="DE" term="%22Collectivism%22">Collectivism</searchLink><br /><searchLink fieldCode="DE" term="%22Individualism%22">Individualism</searchLink><br /><searchLink fieldCode="DE" term="%22Instructional+Design%22">Instructional Design</searchLink><br /><searchLink fieldCode="DE" term="%22Integrated+Curriculum%22">Integrated Curriculum</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s11165-024-10216-y
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0157-244X<br />1573-1898
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This meta-analysis examined whether learning outcomes differ (a) for STEM integration versus traditional instruction and (b) across STEM integration implementations. Based on 79 effect sizes from 40 studies of 15,577 students, those learning via STEM integration outperformed other students on academic achievement tests (g = 0.661; 95% CI [0.548, 0.774]). The effect sizes of STEM integration on achievement were largest for context integration, smaller for content integration, and smallest for tool integration. They were largest for inquiry-based learning, and progressively smaller for problem-based learning, designed-based learning, and project-based learning. They were largest for STEM subject achievement, and progressively smaller for science achievement, math achievement, and engineering achievement. They were larger for collectivist countries than for individualistic countries. Engineering design skills and grade level were not significant moderators. These results can inform integrated STEM instructional design and improve student learning.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1489580
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1489580
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s11165-024-10216-y
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 30
        StartPage: 1273
    Subjects:
      – SubjectFull: Meta Analysis
        Type: general
      – SubjectFull: STEM Education
        Type: general
      – SubjectFull: Academic Achievement
        Type: general
      – SubjectFull: Effect Size
        Type: general
      – SubjectFull: Problem Based Learning
        Type: general
      – SubjectFull: Science Achievement
        Type: general
      – SubjectFull: Mathematics Achievement
        Type: general
      – SubjectFull: Engineering Education
        Type: general
      – SubjectFull: Collectivism
        Type: general
      – SubjectFull: Individualism
        Type: general
      – SubjectFull: Instructional Design
        Type: general
      – SubjectFull: Integrated Curriculum
        Type: general
    Titles:
      – TitleFull: A Meta-Analysis of STEM Integration on Student Academic Achievement
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Shuqi Zhou
      – PersonEntity:
          Name:
            NameFull: Zehua Dong
      – PersonEntity:
          Name:
            NameFull: Hui Hui Wang
      – PersonEntity:
          Name:
            NameFull: Ming Ming Chiu
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0157-244X
            – Type: issn-electronic
              Value: 1573-1898
          Numbering:
            – Type: volume
              Value: 55
            – Type: issue
              Value: 5
          Titles:
            – TitleFull: Research in Science Education
              Type: main
ResultId 1