View in EDS HTML Full Text PDF Full Text

Meta-Analysis of Prevalence: 'I'[superscript 2] Statistic and How to Deal with Heterogeneity

Saved in:
Bibliographic Details
Title: Meta-Analysis of Prevalence: 'I'[superscript 2] Statistic and How to Deal with Heterogeneity
Language: English
Authors: Migliavaca, Celina Borges (ORCID 0000-0003-1389-2311), Stein, Cinara, Colpani, Verônica, Barker, Timothy Hugh, Ziegelmann, Patricia Klarmann, Munn, Zachary, Falavigna, Maicon
Source: Research Synthesis Methods. May 2022 13(3):363-367.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 5
Publication Date: 2022
Document Type: Journal Articles
Information Analyses
Descriptors: Incidence, Meta Analysis, Statistics, Statistical Distributions, Prediction, Data Interpretation, Statistical Analysis
DOI: 10.1002/jrsm.1547
ISSN: 1759-2879
Abstract: Over the last decade, there has been a 10-fold increase in the number of published systematic reviews of prevalence. In meta-analyses of prevalence, the summary estimate represents an average prevalence from included studies. This estimate is truly informative only if there is no substantial heterogeneity among the different contexts being pooled. In systematic reviews, heterogeneity is usually explored with "I"-squared statistic ("I"[superscript 2]), but this statistic does not directly inform us about the distribution of effects and frequently systematic reviewers and readers misinterpret this result. In a sample of 134 meta-analyses of prevalence, the median "I"[superscript 2] was 96.9% (IQR 90.5-98.7). We observed larger "I"[superscript 2] in meta-analysis with higher number of studies and extreme pooled estimates (defined as <10% or >90%). Studies with high "I"[superscript 2] values were more likely to have conducted a sensitivity analysis, including subgroup analysis but only three (2%) systematic reviews reported prediction intervals. We observed that meta-analyses of prevalence often present high "I"[superscript 2] values. However, the number of studies included in the meta-analysis and the point estimate can be associated with the "I"[superscript 2] value, and a high "I"[superscript 2] value is not always synonymous with high heterogeneity. In meta-analyses of prevalence, "I"[superscript 2] statistics may not be discriminative and should be interpreted with caution, avoiding arbitrary thresholds. To discuss heterogeneity, reviewers should focus on the description of the expected range of estimates, which can be done using prediction intervals and planned sensitivity analysis. [This report was written on behalf of the Prevalence Estimates Reviews--Systematic Review Methodology Group (PERSyst).]
Abstractor: As Provided
Entry Date: 2022
Accession Number: EJ1334946
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHi08qu0F7inCFuSAxcpaUQAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDEboyqiHbnIdkjzlsgIBEICBmxS9Yfg9LW_Cvab_Bf_EjZviV9e1rzFL-mXAWeRLdb3X4DmGz_hlk_0b1GcdyGB3LyAgAmBaak7tYqvVY2OsdWuZLWtYUYMSon5lWtg5DrK04Tvaed60iKk8JW8h5f4v6GuwGrFTczcPQGfzxSZChPq0RGM9EFeuXaAnzT2ub-OGuzIdV-_F2NieFW0NykOoB8FV8udv7e0S9wOQ
Text:
  Availability: 1
  Value: &lt;anid&gt;AN0156769198;[bdct]01may.22;2022May11.07:24;v2.2.500&lt;/anid&gt; &lt;title id=&quot;AN0156769198-1&quot;&gt;Meta‐analysis of prevalence: I&amp;lt;sup&amp;gt;2&amp;lt;/sup&amp;gt; statistic and how to deal with heterogeneity&#160;&lt;/title&gt; &lt;p&gt;Over the last decade, there has been a 10‐fold increase in the number of published systematic reviews of prevalence. In meta‐analyses of prevalence, the summary estimate represents an average prevalence from included studies. This estimate is truly informative only if there is no substantial heterogeneity among the different contexts being pooled. In systematic reviews, heterogeneity is usually explored with I‐squared statistic (I2), but this statistic does not directly inform us about the distribution of effects and frequently systematic reviewers and readers misinterpret this result. In a sample of 134 meta‐analyses of prevalence, the median I2 was 96.9% (IQR 90.5–98.7). We observed larger I2 in meta‐analysis with higher number of studies and extreme pooled estimates (defined as &amp;lt;10% or &amp;gt;90%). Studies with high I2 values were more likely to have conducted a sensitivity analysis, including subgroup analysis but only three (2%) systematic reviews reported prediction intervals. We observed that meta‐analyses of prevalence often present high I2 values. However, the number of studies included in the meta‐analysis and the point estimate can be associated with the I2 value, and a high I2 value is not always synonymous with high heterogeneity. In meta‐analyses of prevalence, I2 statistics may not be discriminative and should be interpreted with caution, avoiding arbitrary thresholds. To discuss heterogeneity, reviewers should focus on the description of the expected range of estimates, which can be done using prediction intervals and planned sensitivity analysis.&lt;/p&gt; &lt;p&gt;Keywords: heterogeneity; I2; I‐squared; meta‐analysis; prevalence&lt;/p&gt; &lt;hd id=&quot;AN0156769198-2&quot;&gt;SYSTEMATIC REVIEWS AND META‐ANALYSES OF PREVALENCE&lt;/hd&gt; &lt;p&gt;Over the last decade, there has been a 10‐fold increase in the number of published systematic reviews (SR) that have answered questions of disease prevalence within a population.1 In meta‐analyses of prevalence, the point estimate represents an average prevalence from included studies. This pooled point estimate is truly informative only if there is no substantial variability in the prevalence among the different contexts being pooled; however, this is often an unmet assumption since factors for variability in prevalence are usually well known and expected. Importantly, the definition of &quot;substantial variability&quot; depends on the research question being studied.&lt;/p&gt; &lt;p&gt;Therefore, just as important as the point estimate, the distribution of estimates (i.e., how much they are dispersed around the average pooled estimate) is a key aspect in meta‐analyses of prevalence. This is the heterogeneity of the result, which is frequently (and potentially inappropriately) explored using the I‐squared statistic (&lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt;), not only in meta‐analyses of proportions but also in meta‐analyses of other data types.2&lt;/p&gt; &lt;p&gt;The &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; represents the proportion of the observed variance that cannot be attributed to sampling error.2,3 It reflects the amount to which confidence intervals (CI) for individual study estimates overlap with one another.4–6 As such, when there is significant overlap of CI, &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; will be low. Where there is minimal overlap, &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; will be high. Of note, CI are an index of the precision of the point estimate, related to the sample size of the studies, and are not directly related to heterogeneity. Large sample sizes lead to small CI in individual studies. In a meta‐analysis, they tend not to overlap, leading to high &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; estimates and wide CI for the summary measure.5&lt;/p&gt; &lt;p&gt;Authors mistakenly conclude that low &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; values indicate homogeneity, while high &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; values indicate heterogeneity, and the use of arbitrary thresholds to define which value of &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; is classified as high or low is common.7,8 Nevertheless, this statistic does not directly inform us about the distribution of effects—that is, it does not inform how much the estimates vary around the average pooled estimate or whether the range of estimates is wide or narrow.3 Consequently, high &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; estimates do not necessarily represent a concern in the analysis and are not synonymous with important heterogeneity. Likewise, low values of &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; are not always indicators of consistent and homogenous results.&lt;/p&gt; &lt;hd id=&quot;AN0156769198-3&quot;&gt;I 2 IN META‐ANALYSES OF PREVALENCE&lt;/hd&gt; &lt;p&gt;We assessed the main characteristics of a systematic sample of 235 SR of prevalence published in English and indexed in MEDLINE from 2017 to 2018.1 From those, 152 conducted prevalence meta‐analysis (83 did not conduct) and 134 presented the &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt;. In our analysis, we included the first meta‐analysis presented in these 134 reviews (Figure&#160;S1). More details about the methodology and the main characteristics of the 134 reviews are summarized in the Supporting Information.&lt;/p&gt; &lt;p&gt;One remarkable finding was that the median &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; was 96.9% (interquartile range [IQR] 90.5–98.7). Interestingly, 125 (93.3%) presented &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; ≥ 70% and, as reported previously, 104 (77.6%) presented &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; ≥ 90%.1 This is not common in meta‐analyses for other data types. For instance, a study assessing 1011 Cochrane SR comparing the effect of interventions of any kind and from any health area on binary outcomes found a median &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; of 21.1% (IQR 0.0–49.7), with 5.6% of the analyses with &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; ≥ 75%.9 In another similar example, in a random sample of 137 SR that included clinical studies assessing the effectiveness of interventions from any health field, the average &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; was 20.4% (standard deviation 26.1%).10 Unsurprisingly, systematic reviews with different types of studies have a different distribution of &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt;.4&lt;/p&gt; &lt;p&gt;Considering this finding, we explored the relationship between &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; and variables of interest: number of studies included in the meta‐analysis (classified according to approximate quartiles: 2–9, 10–18, 19–26 and ≥27), pooled estimate (classified into extremes [&amp;lt;10% or &amp;gt;90%] or not [≥10% or ≤90%]) and method used for data transformation (Freeman–Tukey, Others or Not reported). For the analysis, we classified the &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; into high (&amp;gt;50%) and low (≤50%). These thresholds were defined based on what is usually used in the literature, although they are not supported by empirical evidence.7 Of note, these were post‐hoc exploratory analyses and findings need to be validated with additional datasets. Moreover, we assessed how many studies conducted sensitivity analysis, including, but not limited to, subgroup analysis, meta‐regression and test of different data transformation methods, and how many estimated prediction intervals.&lt;/p&gt; &lt;p&gt;The median number of studies included in each meta‐analysis was 19 (IQR 10–27). Meta‐analyses with &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; &amp;gt; 50% included more studies (median 19, IQR 10–28) than meta‐analyses with &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; ≤ 50% (median 9, IQR 6.5–9.5; &lt;emph&gt;p&lt;/emph&gt;&#160;=&#160;0.004, data not shown). All meta‐analyses with more than 19 included studies presented &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; &amp;gt; 50% (Table&#160;1). In the rare in the rare examples where we find low &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; in meta‐analyses of prevalence, it is in analyses with fewer studies. Although &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; does not inherently depend on the number of studies included in a meta‐analysis, when the number of included studies is low, the &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; estimate is biased and it is possible to observe a low &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; by chance.2,11&lt;/p&gt; &lt;p&gt;1 TABLE I 2 assessment according to the variables of interest&lt;/p&gt; &lt;p&gt; &lt;ephtml&gt; &amp;lt;table&amp;gt;&amp;lt;thead valign=&quot;bottom&quot;&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;th align=&quot;left&quot; /&amp;gt;&amp;lt;th align=&quot;left&quot;&amp;gt;Number of systematic reviews&amp;lt;/th&amp;gt;&amp;lt;th align=&quot;left&quot;&amp;gt;&amp;lt;italic&amp;gt;I&amp;lt;/italic&amp;gt;&amp;lt;sup&amp;gt;2&amp;lt;/sup&amp;gt;, Median (IQR)&amp;lt;/th&amp;gt;&amp;lt;th align=&quot;left&quot;&amp;gt;&amp;lt;italic&amp;gt;I&amp;lt;/italic&amp;gt;&amp;lt;sup&amp;gt;2&amp;lt;/sup&amp;gt;&amp;amp;#8201;&amp;amp;#8804;&amp;amp;#8201;50%, &amp;lt;italic&amp;gt;n&amp;lt;/italic&amp;gt; (%)&amp;lt;/th&amp;gt;&amp;lt;th align=&quot;left&quot;&amp;gt;&amp;lt;italic&amp;gt;I&amp;lt;/italic&amp;gt;&amp;lt;sup&amp;gt;2&amp;lt;/sup&amp;gt;&amp;amp;#8201;&amp;amp;#62;&amp;amp;#8201;50%, &amp;lt;italic&amp;gt;n&amp;lt;/italic&amp;gt; (%)&amp;lt;/th&amp;gt;&amp;lt;th align=&quot;left&quot;&amp;gt;&amp;lt;italic&amp;gt;p&amp;lt;/italic&amp;gt;&amp;amp;#8208;Value&amp;lt;/th&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;/thead&amp;gt;&amp;lt;tbody valign=&quot;top&quot;&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;Total&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;134&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;96.9(90.5&amp;amp;#8211;98.7)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;7(5.2%)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;127(94.8%)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;&amp;amp;#8211;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;Number of studies included in the meta&amp;amp;#8208;analysis&amp;lt;xref ref-type=&quot;fn&quot; rid=&quot;tfn2&quot; /&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;td&amp;gt;0.015&amp;lt;xref ref-type=&quot;fn&quot; rid=&quot;tfn3&quot; /&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;02&amp;amp;#8211;09&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;33&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;90.0(75.6&amp;amp;#8211;96.4)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;5(15.2%)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;28(84.8%)&amp;lt;/td&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;10&amp;amp;#8211;18&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;31&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;97.7(89.9&amp;amp;#8211;98.9)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;2(6.4%)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;29(93.6%)&amp;lt;/td&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;19&amp;amp;#8211;26&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;34&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;98.0(95.4&amp;amp;#8211;99.0)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;0&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;34(100%)&amp;lt;/td&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;27 or more&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;36&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;97.1(95.4&amp;amp;#8211;98.9)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;0&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;36(100%)&amp;lt;/td&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;Pooled estimate&amp;lt;/td&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;td&amp;gt;0.015&amp;lt;xref ref-type=&quot;fn&quot; rid=&quot;tfn4&quot; /&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;Between 10% and 90%&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;98&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;97.4(92.6&amp;amp;#8211;98.9)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;2(2.0%)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;96(98.0%)&amp;lt;/td&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;&amp;amp;#60;&amp;amp;#8201;10% or&amp;amp;#8201;&amp;amp;#62;90% (extremes)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;36&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;95.9(85.0&amp;amp;#8211;98.3)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;5 (13.9%)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;31(86.1%)&amp;lt;/td&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;Transformation method&amp;lt;/td&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;td&amp;gt;0.686&amp;lt;xref ref-type=&quot;fn&quot; rid=&quot;tfn3&quot; /&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;Freeman&amp;amp;#8211;Tukey double arcsine&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;32&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;95.7(91.8&amp;amp;#8211;98.7)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;1(3.1%)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;31(96.9%)&amp;lt;/td&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;Others&amp;lt;xref ref-type=&quot;fn&quot; rid=&quot;tfn5&quot; /&amp;gt;&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;10&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;97.6(95.4&amp;amp;#8211;99.0)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;1(10%)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;9(90%)&amp;lt;/td&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;tr&amp;gt;&amp;lt;td&amp;gt;Not reported&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;92&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;96.9(90.3&amp;amp;#8211;98.4)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;5(5.4%)&amp;lt;/td&amp;gt;&amp;lt;td&amp;gt;87(94.6%)&amp;lt;/td&amp;gt;&amp;lt;td /&amp;gt;&amp;lt;/tr&amp;gt;&amp;lt;/tbody&amp;gt;&amp;lt;/table&amp;gt; &lt;/ephtml&gt; &lt;/p&gt; &lt;p&gt;1 &lt;emph&gt;Note&lt;/emph&gt;: &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt;, &lt;emph&gt;I&lt;/emph&gt;‐squared statistic; IQR, interquartile range. &lt;emph&gt;p&lt;/emph&gt;‐Values in bold are statistically significant for an alpha of 0.05.&lt;/p&gt; &lt;ulist&gt; &lt;item&gt;2 a Division by quartiles, approximately.&lt;/item&gt; &lt;item&gt;3 b Pearson&#39;s Chi‐squared test for homogeneity.&lt;/item&gt; &lt;item&gt;4 c Fisher&#39;s exact test.&lt;/item&gt; &lt;item&gt;5 d Other transformation methods included logit, log, arcsine and no transformation (raw).&lt;/item&gt; &lt;/ulist&gt; &lt;p&gt;There was also an association between the summary estimate of the meta‐analysis and the value of &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; (Table&#160;1): meta‐analyses with extreme pooled estimates (defined as &amp;lt;10% or &amp;gt;90%) often associated with a low &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; value. This finding is also expected for proportional data, since the range of potential results is limited to conditions that are either very rare or very common.&lt;/p&gt; &lt;p&gt;We did not observe an association between the value of &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; and the method used for data transformation (Table&#160;1). However, almost 70% of included reviews did not report the transformation method used.&lt;/p&gt; &lt;p&gt;Studies that reported high &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; values were more likely to have conducted a sensitivity analysis. Subgroup analysis was conducted in 82 reviews (61.2%). Only 3 (2.2%) SR reported prediction intervals, as reported previously.1 These are two appropriate methods to explore heterogeneity.&lt;/p&gt; &lt;hd id=&quot;AN0156769198-4&quot;&gt;REFLECTIONS ON THE USE OF I 2 IN META‐ANALYSIS OF PREVALENCE DATA&lt;/hd&gt; &lt;p&gt;It is known that some variables can impact the estimation of &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt;, such as the sample size, the variance estimator, the type of outcome being pooled (e.g., continuous, binary, or proportional), the transformation method used in proportional meta‐analysis and even the outcome itself.5,6,11,12 This happens not only in the context of prevalence studies. However, considering that in most meta‐analyses of prevalence the estimated &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; are high (in contrast with meta‐analyses of other type of data), taking these factors into account during the appraisal of the heterogeneity is even more essential.&lt;/p&gt; &lt;p&gt;Several factors may contribute to high &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; being often observed in meta‐analyses of prevalence. First, prevalence naturally varies according to time, location and subgroup of patients assessed, thus, we may expect important heterogeneity across prevalence studies. Second, often we observe the inclusion of observational studies with very large sample size (e.g., national databases registries), resulting on precise estimates with small variance. Finally, due to the nature of non‐comparative proportional data, we observe more often diverse point estimates among different studies than for comparative measures such as relative risk or odds ratio. For instance, considering two hypothetical prevalence studies with sample size of 200, the first one with 20 events (10%) and the second one with 40 events (20%), the summary measure in a meta‐analysis will result on an &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; of 87%. Now, for two comparative studies with sample size of 200 (both with 100 patients for each arm), first one with 10 and 20 events, and the second with 20 and 40 events for groups A and B respectively, meta‐analysis result will consist of a RR of 0.5 with an &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; of 0%. Note that in both scenarios the study 2 has the double of events than study 1, however inconsistency was present only for the proportional meta‐analysis (Figure&#160;S2). This is also observed in other proportional meta‐analysis types (e.g., incidence).&lt;/p&gt; &lt;p&gt;We also observed that most meta‐analyses conducted sensitivity analysis. We hypothesize that authors who conduct sensitivity analysis aimed to explore the heterogeneity supposedly identified through &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt;. However, since a lower number of included studies is associated with lower &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; values, subgroup analysis may result in low &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt;, which could be inappropriately considered as a &quot;solution&quot; for heterogeneity.&lt;/p&gt; &lt;p&gt;Prediction intervals are more appropriate to evaluate and incorporate uncertainty in the analysis when true heterogeneity is expected. They inform the range of expected estimates—precisely the question of interest when discussing heterogeneity.13 However, as observed, the estimation of prediction intervals is still underused in meta‐analyses of prevalence.&lt;/p&gt; &lt;hd id=&quot;AN0156769198-5&quot;&gt;HOW TO DEAL WITH SUBSTANTIAL HETEROGENEITY IN META‐ANALYSIS OF PREVALENCE?&lt;/hd&gt; &lt;p&gt;True heterogeneity is frequently expected in prevalence estimates due to factors such as the time and place where included studies were conducted and genuine differences in prevalence across populations. If that is the case, heterogeneity should always be incorporated and explored.&lt;/p&gt; &lt;p&gt;The &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; is generally limited when assessing heterogeneity. Where authors still wish to report the &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; or are requested to do so, it should be presented and interpreted with caution. Authors should describe whether high &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; values truly represent relevant heterogeneous estimates according to the research question under assessment. The use of arbitrary thresholds is strongly recommended against.&lt;/p&gt; &lt;p&gt;Moreover, the decision on whether to report the results of a meta‐analysis or not should not be based solely on &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt;. It is worth to note also that the model used for the meta‐analysis (fixed effect or random effects) should not be chosen according to the result of the &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt;.14&lt;/p&gt; &lt;p&gt;In the Supporting Information, we present a case study to illustrate that relying solely on &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; to explore heterogeneity can be misleading.&lt;/p&gt; &lt;p&gt;Even though it is possible to calculate CI for &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt;, they present issues regarding their interpretation. The same problem occurs with other statistics, such as &lt;emph&gt;Q&lt;/emph&gt; or Tau squared.4 If presented, ideally, they should be reported together with a clear description of what they represent in the particular context of a meta‐analysis.&lt;/p&gt; &lt;p&gt;We encourage reviewers to estimate prediction intervals to explore heterogeneity in their analysis. However, it is important to notice that these intervals may be misleading when we have few studies and in the presence of outliers, so the visual inspection of forest plots is always a component of the assessment of inconsistency of results. It is important to notice that CI for the pooled estimate should always be presented, but reviewers must not assume that they reflect the heterogeneity of results.&lt;/p&gt; &lt;p&gt;In a scenario with relevant heterogeneity, it might not be appropriate to pool prevalence estimates. Sensitivity analysis such as meta‐regression and subgroup analysis can be conducted, but always based on previously determined clinical hypotheses to avoid spurious results.&lt;/p&gt; &lt;hd id=&quot;AN0156769198-6&quot;&gt;CONCLUSION&lt;/hd&gt; &lt;p&gt;Meta‐analyses of prevalence frequently yield high &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; values; however, this estimation can be biased, and it is not an index for heterogeneity. High &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; values are not synonymous with important variability between studies and may not be discriminative. Authors and readers should not be overconcerned with high &lt;emph&gt;I&lt;/emph&gt;&lt;sups&gt;2&lt;/sups&gt; values in meta‐analyses of prevalence. To assess heterogeneity in prevalence meta‐analyses, prediction intervals are currently the best available option. In the presence of relevant heterogeneity, sensitivity analyses are also recommended, if planned a priori.&lt;/p&gt; &lt;hd id=&quot;AN0156769198-7&quot;&gt;CONFLICT OF INTEREST&lt;/hd&gt; &lt;p&gt;Zachary Munn is supported by an NHMRC Investigator Grant APP1195676. Other authors declare no interests.&lt;/p&gt; &lt;hd id=&quot;AN0156769198-8&quot;&gt;AUTHOR CONTRIBUTIONS&lt;/hd&gt; &lt;p&gt; &lt;bold&gt;Celina Borges Migliavaca&lt;/bold&gt;: Data curation; formal analysis; writing—original draft. &lt;bold&gt;Cinara Stein&lt;/bold&gt;: Data curation; formal analysis; writing—review &amp;amp; editing. &lt;bold&gt;Ver&#244;nica Colpani&lt;/bold&gt;: Writing—review &amp;amp; editing. &lt;bold&gt;Timothy Hugh Barker&lt;/bold&gt;: Writing—review &amp;amp; editing. &lt;bold&gt;Zachary Munn&lt;/bold&gt;: Writing—review &amp;amp; editing. &lt;bold&gt;Patricia Klarmann Ziegelmann&lt;/bold&gt;: Writing—review &amp;amp; editing. &lt;bold&gt;Maicon Falavigna&lt;/bold&gt;: Supervision; writing—review &amp;amp; editing.&lt;/p&gt; &lt;hd id=&quot;AN0156769198-9&quot;&gt;DATA AVAILABILITY STATEMENT&lt;/hd&gt; &lt;p&gt;The data that support the findings of this study are available from the corresponding author upon reasonable request.&lt;/p&gt; &lt;p&gt;GRAPH: Appendix S1. Supporting Information&lt;/p&gt; &lt;ref id=&quot;AN0156769198-10&quot;&gt; &lt;title&gt; REFERENCES &lt;/title&gt; &lt;blist&gt; &lt;bibl id=&quot;bib1&quot; type=&quot;bt&quot;&gt;1&lt;/bibl&gt; &lt;bibtext&gt; Borges Migliavaca C, Stein C, Colpani V, Barker TH, Munn Z, Falavigna M. How are systematic reviews of prevalence conducted? A methodological study. BMC Med Res Methodol. 2020 ; 20 (1): 96. doi: 10.1186/s12874‐020‐00975‐3&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibl id=&quot;bib2&quot; type=&quot;bt&quot;&gt;2&lt;/bibl&gt; &lt;bibtext&gt; Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta‐analyses. BMJ. 2003 ; 327 (7414): 557 ‐ 560. doi: 10.1136/bmj.327.7414.557&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibl id=&quot;bib3&quot; type=&quot;bt&quot;&gt;3&lt;/bibl&gt; &lt;bibtext&gt; Borenstein M, Higgins JPT, Hedges LV, Rothstein HR. Basics of meta‐analysis: I 2 is not an absolute measure of heterogeneity. Res Synth Methods. 2017 ; 8 (1): 5 ‐ 18. doi: 10.1002/jrsm.1230&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibl id=&quot;bib4&quot; type=&quot;bt&quot;&gt;4&lt;/bibl&gt; &lt;bibtext&gt; Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to meta‐analysis. Wiley ; 2021.&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibl id=&quot;bib5&quot; type=&quot;bt&quot;&gt;5&lt;/bibl&gt; &lt;bibtext&gt; R&#252;cker G, Schwarzer G, Carpenter JR, Schumacher M. Undue reliance on I 2 in assessing heterogeneity may mislead. BMC Med Res Methodol. 2008 ; 8 (1): 79.&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibl id=&quot;bib6&quot; type=&quot;bt&quot;&gt;6&lt;/bibl&gt; &lt;bibtext&gt; Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta‐analysis. Stat Med. 2002 ; 21 (11): 1539 ‐ 1558.&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibl id=&quot;bib7&quot; type=&quot;bt&quot;&gt;7&lt;/bibl&gt; &lt;bibtext&gt; Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). Cochrane; 2021.&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibl id=&quot;bib8&quot; type=&quot;bt&quot;&gt;8&lt;/bibl&gt; &lt;bibtext&gt; Page MJ, Altman DG, McKenzie JE, et al. Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross‐sectional analysis. J Clin Epidemiol. 2018 ; 95 : 7 ‐ 18. doi: 10.1016/j.jclinepi.2017.11.022&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibl id=&quot;bib9&quot; type=&quot;bt&quot;&gt;9&lt;/bibl&gt; &lt;bibtext&gt; Patsopoulos NA, Evangelou E, Ioannidis JP. Sensitivity of between‐study heterogeneity in meta‐analysis: proposed metrics and empirical evaluation. Int J Epidemiol. 2008 ; 37 (5): 1148 ‐ 1157. doi: 10.1093/ije/dyn065&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibtext&gt; Garcia‐Alamino JM, Bankhead C, Heneghan C, Pidduck N, Perera R. Impact of heterogeneity and effect size on the estimation of the optimal information size: analysis of recently published meta‐analyses. BMJ Open. 2017 ; 7 (11): e015888. doi: 10.1136/bmjopen‐2017‐015888&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibtext&gt; von Hippel PT. The heterogeneity statistic I 2 can be biased in small meta‐analyses. BMC Med Res Methodol. 2015 ; 15 : 35. doi: 10.1186/s12874‐015‐0024‐z&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibtext&gt; Alba AC, Alexander PE, Chang J, MacIsaac J, DeFry S, Guyatt GH. High statistical heterogeneity is more frequent in meta‐analysis of continuous than binary outcomes. J Clin Epidemiol. 2016 ; 70 : 129 ‐ 135. doi: 10.1016/j.jclinepi.2015.09.005&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibtext&gt; IntHout J, Ioannidis JPA, Rovers MM, Goeman JJ. Plea for routinely presenting prediction intervals in meta‐analysis. BMJ Open. 2016 ; 6 (7): e010247. doi: 10.1136/bmjopen‐2015‐010247&lt;/bibtext&gt; &lt;/blist&gt; &lt;blist&gt; &lt;bibtext&gt; Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed‐effect and random‐effects models for meta‐analysis. Res Synth Methods. 2010 ; 1 (2): 97 ‐ 111. doi: 10.1002/jrsm.12&lt;/bibtext&gt; &lt;/blist&gt; &lt;/ref&gt; &lt;aug&gt; &lt;p&gt;By Celina Borges Migliavaca; Cinara Stein; Ver&#244;nica Colpani; Timothy Hugh Barker; Patricia Klarmann Ziegelmann; Zachary Munn and Maicon Falavigna&lt;/p&gt; &lt;p&gt;Reported by Author; Author; Author; Author; Author; Author; Author&lt;/p&gt; &lt;/aug&gt;
Header DbId: eric
DbLabel: ERIC
An: EJ1334946
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Meta-Analysis of Prevalence: &#39;I&#39;[superscript 2] Statistic and How to Deal with Heterogeneity
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: &lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Migliavaca%2C+Celina+Borges%22&quot;&gt;Migliavaca, Celina Borges&lt;/searchLink&gt; (ORCID &lt;externalLink term=&quot;https://orcid.org/0000-0003-1389-2311&quot;&gt;0000-0003-1389-2311&lt;/externalLink&gt;)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Stein%2C+Cinara%22&quot;&gt;Stein, Cinara&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Colpani%2C+Ver&#244;nica%22&quot;&gt;Colpani, Ver&#244;nica&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Barker%2C+Timothy+Hugh%22&quot;&gt;Barker, Timothy Hugh&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Ziegelmann%2C+Patricia+Klarmann%22&quot;&gt;Ziegelmann, Patricia Klarmann&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Munn%2C+Zachary%22&quot;&gt;Munn, Zachary&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Falavigna%2C+Maicon%22&quot;&gt;Falavigna, Maicon&lt;/searchLink&gt;
– Name: TitleSource
  Label: Source
  Group: Src
  Data: &lt;searchLink fieldCode=&quot;SO&quot; term=&quot;%22Research+Synthesis+Methods%22&quot;&gt;&lt;i&gt;Research Synthesis Methods&lt;/i&gt;&lt;/searchLink&gt;. May 2022 13(3):363-367.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley &amp; Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 5
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles&lt;br /&gt;Information Analyses
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: &lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Incidence%22&quot;&gt;Incidence&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Meta+Analysis%22&quot;&gt;Meta Analysis&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Statistics%22&quot;&gt;Statistics&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Statistical+Distributions%22&quot;&gt;Statistical Distributions&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Prediction%22&quot;&gt;Prediction&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Data+Interpretation%22&quot;&gt;Data Interpretation&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Statistical+Analysis%22&quot;&gt;Statistical Analysis&lt;/searchLink&gt;
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1002/jrsm.1547
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1759-2879
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Over the last decade, there has been a 10-fold increase in the number of published systematic reviews of prevalence. In meta-analyses of prevalence, the summary estimate represents an average prevalence from included studies. This estimate is truly informative only if there is no substantial heterogeneity among the different contexts being pooled. In systematic reviews, heterogeneity is usually explored with &quot;I&quot;-squared statistic (&quot;I&quot;[superscript 2]), but this statistic does not directly inform us about the distribution of effects and frequently systematic reviewers and readers misinterpret this result. In a sample of 134 meta-analyses of prevalence, the median &quot;I&quot;[superscript 2] was 96.9% (IQR 90.5-98.7). We observed larger &quot;I&quot;[superscript 2] in meta-analysis with higher number of studies and extreme pooled estimates (defined as &lt;10% or &gt;90%). Studies with high &quot;I&quot;[superscript 2] values were more likely to have conducted a sensitivity analysis, including subgroup analysis but only three (2%) systematic reviews reported prediction intervals. We observed that meta-analyses of prevalence often present high &quot;I&quot;[superscript 2] values. However, the number of studies included in the meta-analysis and the point estimate can be associated with the &quot;I&quot;[superscript 2] value, and a high &quot;I&quot;[superscript 2] value is not always synonymous with high heterogeneity. In meta-analyses of prevalence, &quot;I&quot;[superscript 2] statistics may not be discriminative and should be interpreted with caution, avoiding arbitrary thresholds. To discuss heterogeneity, reviewers should focus on the description of the expected range of estimates, which can be done using prediction intervals and planned sensitivity analysis. [This report was written on behalf of the Prevalence Estimates Reviews--Systematic Review Methodology Group (PERSyst).]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2022
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1334946
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1334946
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1002/jrsm.1547
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 5
        StartPage: 363
    Subjects:
      – SubjectFull: Incidence
        Type: general
      – SubjectFull: Meta Analysis
        Type: general
      – SubjectFull: Statistics
        Type: general
      – SubjectFull: Statistical Distributions
        Type: general
      – SubjectFull: Prediction
        Type: general
      – SubjectFull: Data Interpretation
        Type: general
      – SubjectFull: Statistical Analysis
        Type: general
    Titles:
      – TitleFull: Meta-Analysis of Prevalence: 'I'[superscript 2] Statistic and How to Deal with Heterogeneity
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Migliavaca, Celina Borges
      – PersonEntity:
          Name:
            NameFull: Stein, Cinara
      – PersonEntity:
          Name:
            NameFull: Colpani, Verônica
      – PersonEntity:
          Name:
            NameFull: Barker, Timothy Hugh
      – PersonEntity:
          Name:
            NameFull: Ziegelmann, Patricia Klarmann
      – PersonEntity:
          Name:
            NameFull: Munn, Zachary
      – PersonEntity:
          Name:
            NameFull: Falavigna, Maicon
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 05
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-print
              Value: 1759-2879
          Numbering:
            – Type: volume
              Value: 13
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Research Synthesis Methods
              Type: main
ResultId 1