View in EDS HTML Full Text PDF Full Text

Random-Effects Meta-Analysis of Few Studies Involving Rare Events

Saved in:
Bibliographic Details
Title: Random-Effects Meta-Analysis of Few Studies Involving Rare Events
Language: English
Authors: Günhan, Burak Kürsad (ORCID 0000-0002-7454-8680), Röver, Christian (ORCID 0000-0002-6911-698X), Friede, Tim (ORCID 0000-0001-5347-7441)
Source: Research Synthesis Methods. Jan 2020 11(1):74-90.
Availability: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Peer Reviewed: Y
Page Count: 17
Publication Date: 2020
Document Type: Journal Articles
Information Analyses
Descriptors: Meta Analysis, Medical Research, Drug Therapy, Bayesian Statistics, Probability, Data Analysis, Databases, Simulation, Statistical Bias, Intervals, Safety, Pediatrics, Surgery, Physiology, Computer Software, Models
DOI: 10.1002/jrsm.1370
ISSN: 1759-2879
Abstract: Meta-analyses of clinical trials targeting rare events face particular challenges when the data lack adequate numbers of events for all treatment arms. Especially when the number of studies is low, standard random-effects meta-analysis methods can lead to serious distortions because of such data sparsity. To overcome this, we suggest the use of "weakly informative priors" (WIPs) for the treatment effect parameter of a Bayesian meta-analysis model, which may also be seen as a form of penalization. As a data model, we use a binomial-normal hierarchical model (BNHM) that does not require continuity corrections in case of zero counts in one or both arms. We suggest a normal prior for the log-odds ratio with mean 0 and standard deviation 2.82, which is motivated (a) as a symmetric prior centered around unity and constraining the odds ratio within a range from 1/250 to 250 with 95% probability and (b) as consistent with empirically observed effect estimates from a set of 37 773 meta-analyses from the Cochrane Database of Systematic Reviews. In a simulation study with rare events and few studies, our BNHM with a WIP outperformed a Bayesian method without a WIP and a maximum likelihood estimator in terms of smaller bias and shorter interval estimates with similar coverage. Furthermore, the methods are illustrated by a systematic review in immunosuppression of rare safety events following pediatric transplantation. A publicly available R package, MetaStan, is developed to automate a Bayesian implementation of meta-analysis models using WIPs.
Abstractor: As Provided
Entry Date: 2020
Accession Number: EJ1253063
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFcvycvynx8Os-HFJs5I8vTAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDO6BMWP0OC8h4T9wowIBEICBm4FKN5BYv7rL1XBDuAHZlKR2Bxqv28trpVzhWVNPwQE_iF-RrHl1xwMCy6u2xd2wlK42Vlc5CuJzIDd5MShnPgLq8VfgHjej9ynxRW6_-4IPXYrPHWeAsFV_fKwoakxAmCcyQWinQiwZ6IEySEq87pSomKTnGVfqPn9f2VQl1Y-GcCn5W54KRZJrBHlzEVn6IoMKIGgAhD7fDiun
Text:
  Availability: 1
  Value: <anid>AN0141251721;[bdct]01jan.20;2020Jan20.02:13;v2.2.500</anid> <title id="AN0141251721-1">Random‐effects meta‐analysis of few studies involving rare events </title> <p>Meta‐analyses of clinical trials targeting rare events face particular challenges when the data lack adequate numbers of events for all treatment arms. Especially when the number of studies is low, standard random‐effects meta‐analysis methods can lead to serious distortions because of such data sparsity. To overcome this, we suggest the use of weakly informative priors (WIPs) for the treatment effect parameter of a Bayesian meta‐analysis model, which may also be seen as a form of penalization. As a data model, we use a binomial‐normal hierarchical model (BNHM) that does not require continuity corrections in case of zero counts in one or both arms. We suggest a normal prior for the log‐odds ratio with mean 0 and standard deviation 2.82, which is motivated (a) as a symmetric prior centered around unity and constraining the odds ratio within a range from 1/250 to 250 with 95% probability and (b) as consistent with empirically observed effect estimates from a set of 37 773 meta‐analyses from the Cochrane Database of Systematic Reviews. In a simulation study with rare events and few studies, our BNHM with a WIP outperformed a Bayesian method without a WIP and a maximum likelihood estimator in terms of smaller bias and shorter interval estimates with similar coverage. Furthermore, the methods are illustrated by a systematic review in immunosuppression of rare safety events following pediatric transplantation. A publicly available R package, MetaStan, is developed to automate a Bayesian implementation of meta‐analysis models using WIPs.</p> <p>Keywords: Bayes; few studies; random‐effects meta‐analysis; rare events; weakly informative priors</p> <hd id="AN0141251721-2">1 INTRODUCTION</hd> <p>Individual clinical studies are often underpowered to detect difference of probabilities or rates of rare events, for example, safety events, and thus, meta‐analysis may be the only way to obtain reliable evidence of treatment differences with regard to the rare events.[<reflink idref="bib1" id="ref1">1</reflink>] On the other hand, meta‐analysis of clinical studies for rare events faces particular challenges, since the numbers of events might be very small in some treatment arms. The problem is even more pronounced when some studies have no events either in one or in both treatment arms (so‐called single‐zero or double‐zero studies).</p> <p>The exclusion of the double‐zero studies from the analysis can bias the treatment effect parameter estimate away from the null (especially for the unbalanced design)[<reflink idref="bib2" id="ref2">2</reflink>] and also causes loss of information, since double‐zero studies contain information through their sample sizes.[<reflink idref="bib3" id="ref3">3</reflink>] Hence, we consider methods that do not remove double‐zero studies from the analysis. Two established fixed‐effect meta‐analysis methods exist for rare events, namely, Peto's method[<reflink idref="bib4" id="ref4">4</reflink>] and the Mantel‐Haenszel (MH) method.[<reflink idref="bib5" id="ref5">5</reflink>] On the other hand, an assumption of homogeneity, that is, a single common parameter for all studies, is typically unrealistic for studies in the biomedical sciences.[[<reflink idref="bib6" id="ref6">6</reflink>]] Therefore, we focus on random‐effects methods in this paper.</p> <p>Standard (approximate) random‐effects meta‐analysis methods, for example, the normal‐normal hierarchical model,[<reflink idref="bib9" id="ref7">9</reflink>] require a <emph>continuity correction</emph> in case of single‐zero or double‐zero studies, that is, the addition of a fixed value (typically 0.5) to all cells of the contingency table for studies with no events or with 100% events (no nonevents). Such simple approaches have been found problematic for meta‐analyses involving rare events.[<reflink idref="bib10" id="ref8">10</reflink>] Therefore, statistical models based on exact distributional assumptions have been suggested. These include different parametrizations of the binomial‐normal hierarchical model (BNHM),[<reflink idref="bib11" id="ref9">11</reflink>] a mixed effects conditional logistic model,[<reflink idref="bib12" id="ref10">12</reflink>] a Poisson‐normal hierarchical model,[<reflink idref="bib13" id="ref11">13</reflink>] a Poisson‐Gamma hierarchical model,[<reflink idref="bib14" id="ref12">14</reflink>] or a beta‐binomial model (BBM).[<reflink idref="bib3" id="ref13">3</reflink>] In this paper, we focus on a parametrization of the BNHM that was suggested by Smith et al.[<reflink idref="bib15" id="ref14">15</reflink>]</p> <p>Consider an extreme case of meta‐analysis of rare events, where all studies include no events for the same treatment arm. These data sparsity problem in a meta‐analysis can be seen as a <emph>separation</emph> problem in the logistic regression context[<reflink idref="bib16" id="ref15">16</reflink>] in which case a maximum likelihood estimate (MLE) for the treatment effect parameter does not exist. A very useful way to deal with separation problems, or, more generally, data sparsity in logistic regression is <emph>penalization</emph>, that is, adding a penalty (adjustment) term to the original likelihood function to regularize (or stabilize) the estimates.[<reflink idref="bib17" id="ref16">17</reflink>] In a frequentist framework, penalty terms may be specified so that these <emph>nudge</emph> the MLE into a desired direction if the maximum is not or poorly defined; one such example is Firth penalization.[[<reflink idref="bib17" id="ref17">17</reflink>]] From a Bayesian viewpoint, penalization may often be motivated as <emph>weakly informative priors</emph> (WIPs) that are multiplied to the likelihood function.[<reflink idref="bib20" id="ref18">20</reflink>]</p> <p>Numbers of studies included in meta‐analyses are typically small, posing additional challenges.[<reflink idref="bib21" id="ref19">21</reflink>] For Bayesian meta‐analysis of few studies, different WIPs have been suggested for the heterogeneity parameter; see Chung et al[<reflink idref="bib22" id="ref20">22</reflink>] for penalized MLE approach and also see Gelman[<reflink idref="bib23" id="ref21">23</reflink>] and Friede and Röver[<reflink idref="bib24" id="ref22">24</reflink>] for Bayesian inference. Here, we consider the meta‐analysis of few studies targeting rare events. To deal with data sparsity present in the meta‐analysis of few studies with rare events, we suggest the use of WIPs for the treatment effect parameter in a fully Bayesian context inspired by penalization ideas.[[<reflink idref="bib17" id="ref23">17</reflink>], [<reflink idref="bib20" id="ref24">20</reflink>]] We use a BNHM that is parameterized in terms of baseline risks and a treatment effect for the data. Note that this is a contrast‐based model meaning that relative treatment effects are assumed to be exchangeable across trials.[<reflink idref="bib25" id="ref25">25</reflink>] Our suggested default WIP for the treatment effect parameter is motivated via the consideration of the prior expected range of treatment effect values. Furthermore, it is consistent with effect estimates empirically observed in a large set of meta‐analyses from the Cochrane Database of Systematic Reviews (CDSR) with binary endpoints.</p> <p>The main contribution of this paper is the introduction of default WIPs as penalization for treatment effect parameters to deal with data sparsity in the meta‐analysis of few studies involving rare events. Another contribution is the introduction of an <bold>R</bold> package, <bold>MetaStan</bold>(https://CRAN.R-project.org/package=MetaStan), which is developed to automate a Bayesian implementation of meta‐analysis models using WIPs as described in the paper and which is publicly available from CRAN. In Section , we describe a systematic review concerning rare safety events associated with immunosuppressive therapy following pediatric transplantation. In Section , we describe the application of WIPs for the treatment effect parameter. We review a BNHM for meta‐analysis, discuss the derivation of WIP, and an empirical investigation of treatment effect parameter estimates from the CDSR. Long‐run properties of different methods including the proposed one are investigated in the simulation studies in Section. In Section , the example is revisited to illustrate the proposed method and its implementation. We close with some conclusions and provide a discussion.</p> <hd id="AN0141251721-3">2 AN APPLICATION IN PEDIATRIC TRANSPLANTATION</hd> <p>Several rare pediatric liver diseases can nowadays be successfully treated by liver transplantation with good long‐term outcomes. Crins et al[<reflink idref="bib26" id="ref26">26</reflink>] conducted a systematic review of controlled but not necessarily randomized studies of the Interleukin‐2 receptor antibodies (IL‐2RA) basiliximab and daclizumab in pediatric liver transplantation. Primary outcomes were acute rejections (ARs), steroid‐resistant rejections (SRRs), graft loss, and death. Their analyses were based on a random‐effects meta‐analysis using a restricted maximum likelihood approach(REML).[<reflink idref="bib27" id="ref27">27</reflink>] Crins et al[<reflink idref="bib26" id="ref28">26</reflink>] used risk ratios as effect measures, while we use the odds ratios here. With rare events, however, these should be very similar. Heterogeneity was assessed using Cochrane's Q test.[<reflink idref="bib28" id="ref29">28</reflink>] Secondary outcomes included renal dysfunction nt lymphoproliferative disease (PTLD). For illustrative purposes, here, we focus on death and PTLDs, and these outcomes are displayed in Table.</p> <p>Data on patient deaths and posttransplant lymphoproliferative disease (PTLD) from the meta‐analysis in pediatric transplantation conducted by Crins et al</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left"> Outcome: Death</th><th align="left"> Outcome: PTLD</th></tr><tr><th align="left" /><th align="left"> Control</th><th align="left"> Experimental</th><th align="left"> Control</th><th align="left"> Experimental</th></tr><tr><th align="left" /><th align="left">Events</th><th align="left">Total</th><th align="left">Events</th><th align="left">Total</th><th align="left">Events</th><th align="left">Total</th><th align="left">Events</th><th align="left">Total</th></tr></thead><tbody valign="top"><tr><td align="left">Heffron et al<xref ref-type="bibr" rid="bibr29">29</xref></td><td align="left">3</td><td align="left">20</td><td align="left">4</td><td align="left">61</td><td align="left">‐</td><td align="left">‐</td><td align="left">‐</td><td align="left">‐</td></tr><tr><td align="left">Schuller et al<xref ref-type="bibr" rid="bibr30">30</xref></td><td align="left">‐</td><td align="left">‐</td><td align="left">‐</td><td align="left">‐</td><td align="left">0</td><td align="left">12</td><td align="left">0</td><td align="left">18</td></tr><tr><td align="left">Ganschow et al<xref ref-type="bibr" rid="bibr31">31</xref></td><td align="left">3</td><td align="left">54</td><td align="left">1</td><td align="left">54</td><td align="left">0</td><td align="left">54</td><td align="left">1</td><td align="left">54</td></tr><tr><td align="left">Spada et al<xref ref-type="bibr" rid="bibr32">32</xref></td><td align="left">3</td><td align="left">36</td><td align="left">4</td><td align="left">36</td><td align="left">1</td><td align="left">36</td><td align="left">1</td><td align="left">36</td></tr><tr><td align="left">Gras et al<xref ref-type="bibr" rid="bibr33">33</xref></td><td align="left">3</td><td align="left">34</td><td align="left">2</td><td align="left">50</td><td align="left">‐</td><td align="left">‐</td><td align="left">‐</td><td align="left">‐</td></tr></tbody></table> </ephtml> </p> <p>The specific problems with meta‐analyses concerning rare events outlined in the introduction are prominent here. Firstly, the numbers of events are very small. For the PTLD dataset, there is one single‐zero study and one double‐zero study. Secondly, there are few studies available, only four for deaths and three for PTLD. Empirical event rates are lower in three of the four experimental groups in the data on patient deaths. For PTLD, the data appear inconclusive for Schuller et al[<reflink idref="bib30" id="ref30">30</reflink>] and Spada et al,[<reflink idref="bib32" id="ref31">32</reflink>] and only a single event observed in the experimental group suggests an increased risk in the study by Ganschow et al.[<reflink idref="bib31" id="ref32">31</reflink>]</p> <hd id="AN0141251721-4">3 WIPS FOR THE TREATMENT EFFECT</hd> <p>In this section, we present the usage of WIPs for the treatment effect parameter to conduct random‐effects meta‐analysis of rare events with few studies. As a data model, we review a BNHM and then show how to derive a WIP for a treatment effect parameter. Then, empirical evidence obtained from the CDSR supporting the choice of WIPs is illustrated.</p> <hd id="AN0141251721-5">3.1 Data model</hd> <p>The BNHM has been introduced by Smith et al.[<reflink idref="bib15" id="ref33">15</reflink>] In the BNHM, for each trial <emph>i</emph>∈{1,...,<emph>k</emph>} and treatment arm <emph>j</emph>∈{0,1}, the event counts <emph>r</emph><subs><emph>ij</emph></subs> are modeled using a binomial distribution, that is, <emph>r</emph><subs><emph>ij</emph></subs>∼Bin(<emph>π</emph><subs><emph>ij</emph></subs>,<emph>n</emph><subs><emph>ij</emph></subs>). The logit link is used to transform <emph>π</emph><subs><emph>ij</emph></subs> onto the log odds scale where effects can be assumed to be additive</p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0001" xmlns="http://www.w3.org/1998/Math/MathML">logit(πij)=μi+θixijθi∼N(θ,τ2),</math> </ephtml> </p> <p>where <emph>x</emph><subs><emph>ij</emph></subs> is a treatment indicator, namely, +0.5 = experimental (<emph>j</emph> = 1) and −0.5 = control (<emph>j</emph> = 0). The <emph>μ</emph><subs><emph>i</emph></subs> are fixed effects denoting the baseline risks of the event in each study <emph>i</emph>, <emph>θ</emph> is the mean treatment effect, and <emph>τ</emph> is the heterogeneity in treatment effects between trials. The BNHM belongs to the family of generalized linear mixed models (GLMMs); this family also includes models for other types of data including continuous or count outcomes. It is important to note here that by treating the baseline risks <emph>μ</emph><subs><emph>i</emph></subs> as fixed effects, the analysis effectively stratifies the risk by study, as pooling of risks might compromise the studies' randomization. In this sense, it constitutes a contrast‐based model.[<reflink idref="bib25" id="ref34">25</reflink>] Unlike the normal‐normal hierarchical model, the BNHM does not rely on a normal approximation, since it builds on the binomial nature of the data directly.</p> <p>The BNHM can be fitted using frequentist approaches, for example, via maximum likelihood estimation (MLE).[<reflink idref="bib11" id="ref35">11</reflink>] Alternatively, Bayesian methods are commonly used. In a fully Bayesian approach, prior distributions for parameters <emph>θ</emph>, <emph>μ</emph><subs><emph>i</emph></subs>, and <emph>τ</emph> need to be specified. Note that the parameter <emph>θ</emph> is on the log‐odds ratio scale whereas <emph>μ</emph><subs><emph>i</emph></subs> are on the log‐odds scale. Baseline risks (<emph>μ</emph><subs><emph>i</emph></subs>) may be seen as intercept parameters in a standard logistic regression model. For <emph>μ</emph><subs><emph>i</emph></subs>, we use a vague normal prior with mean 0 and standard deviation 10, following the recommendation by Gelman et al.[<reflink idref="bib20" id="ref36">20</reflink>] The prior choice for <emph>θ</emph> is our main focus and will be discussed in Section. The prior choice for the heterogeneity parameter <emph>τ</emph>, which is a standard deviation parameter, has gained much attention in the literature as discussed in the introduction. Friede et al[<reflink idref="bib24" id="ref37">24</reflink>] have shown that for meta‐analysis of few studies, the use of WIPs for <emph>τ</emph> displays desirable long‐run properties in comparison with frequentist alternatives. Following their suggestions, we use a half‐normal prior with scale of 0.5 ( <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0002" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">H</mi><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>.</mo><mn>5</mn><mo>)</mo></math> </ephtml> ) for <emph>τ</emph> which has the median of 0.337 with an upper 95% quantile of 0.98. Values for <emph>τ</emph> of 0.25, 0.5, 1, and 2 represent moderate, substantial, large, and very large heterogeneity.[<reflink idref="bib34" id="ref38">34</reflink>] Thus, a <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0003" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">H</mi><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>.</mo><mn>5</mn><mo>)</mo></math> </ephtml> prior captures heterogeneity values for log‐OR typically seen in meta‐analyses of log‐ORs and will therefore be a sensible choice in many applications.</p> <hd id="AN0141251721-6">3.2 Derivation of a WIP for the treatment effect</hd> <p>A common prior choice for the treatment effect parameter <emph>θ</emph> is a noninformative (vague) prior such as normal distribution with a large variance, for example, <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0004" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>,</mo><mn>10</mn><msup><mn>0</mn><mn>2</mn></msup><mo>)</mo></math> </ephtml> . One way of constructing a WIP works via consideration of the prior expected range of treatment effect values.[<reflink idref="bib35" id="ref39">35</reflink>] Before the derivation of the WIP for treatment effect parameter <emph>θ</emph>, recall that <emph>θ</emph> is on the log‐odds ratio scale. Thus, a value of <emph>θ</emph>=0 means an odds ratio of 1, ie, <emph>no effect</emph>, and a value of <emph>θ</emph>=1 means that odds differ by a factor (ratio) of exp(<reflink idref="bib1" id="ref40">1</reflink>)=2.7.</p> <p>We assume a symmetric prior centered around zero, implying equal probabilities for positive or negative treatment effects. Symmetry then implies (on the log‐odds ratio scale) that</p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0005" xmlns="http://www.w3.org/1998/Math/MathML"><mrow /></math> </ephtml> </p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0006" xmlns="http://www.w3.org/1998/Math/MathML"><mi>P</mi><mo>(</mo><mi>θ</mi><mo>></mo><mi>q</mi><mo>)</mo><mo>=</mo><mi>P</mi><mo>(</mo><mi>θ</mi><mo><</mo><mo>−</mo><mi>q</mi><mo>)</mo><mo>,</mo></math> </ephtml> </p> <p>where (on the odds ratio scale)</p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0007" xmlns="http://www.w3.org/1998/Math/MathML"><mrow /></math> </ephtml> </p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0008" xmlns="http://www.w3.org/1998/Math/MathML"><mi>exp</mi><mo>(</mo><mo>−</mo><mi>q</mi><mo>)</mo><mo>=</mo><mfrac><mn>1</mn><mrow><mi>exp</mi><mo>(</mo><mi>q</mi><mo>)</mo></mrow></mfrac><mo>.</mo></math> </ephtml> </p> <p>The prior's scale parameter <emph>σ</emph><subs>prior</subs> then may be set such that a priori the odds ratio is with 95% probability confined to a certain range:</p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0009" xmlns="http://www.w3.org/1998/Math/MathML"><mrow /></math> </ephtml> </p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0010" xmlns="http://www.w3.org/1998/Math/MathML"><mi>P</mi><mo>(</mo><mn>1</mn><mo>/</mo><mi>δ</mi><mo><</mo><mi>exp</mi><mo>(</mo><mi>θ</mi><mo>)</mo><mo><</mo><mi>δ</mi><mo>)</mo><mo>=</mo><mn>95</mn><mi>%</mi><mo>.</mo></math> </ephtml> </p> <p>In case of a normal prior with standard deviation <emph>σ</emph><subs>prior</subs>, we can then simply specify</p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0011" xmlns="http://www.w3.org/1998/Math/MathML"><mrow /></math> </ephtml> </p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0012" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>σ</mi><mtext>prior</mtext></msub><mo>=</mo><mfrac><mrow><mi>log</mi><mo>(</mo><mi>δ</mi><mo>)</mo></mrow><mrow><mn>1</mn><mo>.</mo><mn>96</mn></mrow></mfrac><mo>.</mo></math> </ephtml> </p> <p>We conservatively specify <emph>δ</emph> as 250, meaning that we consider it unlikely that the odds ratio will be larger than 250 or smaller than 1/250. By plugging in this number into , we obtain <emph>σ</emph><subs>prior</subs>=2.82.</p> <p>Another way to motivate the prior standard deviation is by using the idea of <emph>unit information priors</emph>.[[<reflink idref="bib36" id="ref41">36</reflink>]] When the treatment effect parameter is on the log‐odds ratio scale (as in the BNHM), then the standard error is given by <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0013" xmlns="http://www.w3.org/1998/Math/MathML"><msqrt><mrow><mfrac><mn>1</mn><mi>a</mi></mfrac><mo>+</mo><mfrac><mn>1</mn><mi>b</mi></mfrac><mo>+</mo><mfrac><mn>1</mn><mi>c</mi></mfrac><mo>+</mo><mfrac><mn>1</mn><mi>d</mi></mfrac></mrow></msqrt></math> </ephtml> . Assuming equal allocation, a neutral effect, and equal counts of events and nonevents, we can simply set the table allocation to <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0014" xmlns="http://www.w3.org/1998/Math/MathML"><mi>a</mi><mo>=</mo><mi>b</mi><mo>=</mo><mi>c</mi><mo>=</mo><mi>d</mi><mo>=</mo><mfrac><mi>N</mi><mn>4</mn></mfrac></math> </ephtml> . Therefore, if we (heuristically) reverse the argument, a prior for the log‐odds ratio with zero mean and 2.82 standard deviation gives[<reflink idref="bib37" id="ref42">37</reflink>]</p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0015" xmlns="http://www.w3.org/1998/Math/MathML"><mrow /></math> </ephtml> </p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0016" xmlns="http://www.w3.org/1998/Math/MathML"><mn>2</mn><mo>.</mo><mn>82</mn><mo>≈</mo><msqrt><mn>8</mn></msqrt><mo>=</mo><msqrt><mrow><mfrac><mn>1</mn><mrow><mfrac><mi>N</mi><mn>4</mn></mfrac></mrow></mfrac><mo>+</mo><mfrac><mn>1</mn><mrow><mfrac><mi>N</mi><mn>4</mn></mfrac></mrow></mfrac><mo>+</mo><mfrac><mn>1</mn><mrow><mfrac><mi>N</mi><mn>4</mn></mfrac></mrow></mfrac><mo>+</mo><mfrac><mn>1</mn><mrow><mfrac><mi>N</mi><mn>4</mn></mfrac></mrow></mfrac></mrow></msqrt><mo>.</mo></math> </ephtml> </p> <p>Hence, <emph>N</emph>=2. In other words, in terms of prior's effective sample size, the choice of <emph>σ</emph><subs>prior</subs>=2.82 is equivalent to adding two patients to the dataset. From this, it follows that <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0017" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>,</mo><mn>2</mn><mo>.</mo><mn>8</mn><msup><mn>2</mn><mn>2</mn></msup><mo>)</mo></math> </ephtml> is a reasonable choice as a WIP for <emph>θ</emph>.</p> <p>Note also the analogy between this WIP and commonly used continuity corrections: Zero entries in a contingency tables are commonly <emph> fixed </emph> by adding a correction term of 0.5 to each table cell of the single‐zero or double‐zero study, which also amounts to a total of two patients <emph> added </emph> to the data. This way of conducting continuity correction adds two patients to each single‐zero or double‐zero study, while the use of WIP is equivalent to adding two patients to the whole dataset.</p> <hd id="AN0141251721-7">3.3 Empirical evidence supporting the WIP for the treatment effect</hd> <p>For an empirical investigation of the WIP for treatment effect parameter, we consider the meta‐analysis datasets archived in the CDSR. All systematic reviews in the CDSR are available on the Cochrane Library website,[<reflink idref="bib38" id="ref43">38</reflink>] and personal or institutional access is required. For downloading the data from the CDSR and converting to CSV files, we use the program Cochrane_scraper (version 1.1.0).[<reflink idref="bib39" id="ref44">39</reflink>] We were able to access all Cochrane systematic reviews available in March 2018 (CD000004 to CD012788). Meta‐analyses were excluded if they included only one study, if the analysis was labelled as a subgroup or sensitivity analysis or there was insufficient information for classification, or if all data within the meta‐analysis appeared to be erroneous. Finally, we only consider meta‐analyses with dichotomous outcomes. In total, 37 773 meta‐analysis datasets from 4712 reviews are included. Note that we did not distinguish regarding efficacy or safety analyses.</p> <p>The frequency of the number of studies <emph>k</emph> considered for each meta‐analysis is illustrated in Figure. The percentage of the meta‐analyses including five or less studies is 66%. This figure is consistent with other re‐analyses of the CDSR (see, eg, previous works[[<reflink idref="bib8" id="ref45">8</reflink>], [<reflink idref="bib21" id="ref46">21</reflink>], [<reflink idref="bib40" id="ref47">40</reflink>]]). We re‐analyzed the meta‐analysis datasets from the CDSR using the BNHM via an MLE approach. This procedure is implemented using the <bold>R</bold> package lme4.[<reflink idref="bib41" id="ref48">41</reflink>] A histogram of the estimates of <emph>θ</emph> is illustrated in Figure A; 2.5% and 97.5% quantiles of the estimates of <emph>θ</emph> are −1.94 and 2.06, respectively. By following Turner et al,[<reflink idref="bib40" id="ref49">40</reflink>] we exclude the zero heterogeneity estimates; nonzero estimates of <emph>τ</emph> are shown in Figure B. The fraction of nonzero heterogeneity estimates is 63%, which is also consistent with previous findings.[<reflink idref="bib40" id="ref50">40</reflink>] The 95% quantile of nonzero estimates of <emph>τ</emph> is 1.51, while the 95% quantile of <emph>τ</emph> estimates including zeroes is 1.05. The underlying distribution of the estimates of <emph>θ</emph> and <emph>τ</emph> and their variability are useful to see how large these estimates are in some general population, in this case the CDSR. Thus, these give us a rough sense of what would be a reasonable default prior distribution. Therefore, we suggest the use of WIPs, <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0018" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>,</mo><mn>2</mn><mo>.</mo><mn>8</mn><msup><mn>2</mn><mn>2</mn></msup><mo>)</mo></math> </ephtml> for <emph>θ</emph> and <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0019" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">H</mi><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>.</mo><mn>5</mn><mo>)</mo></math> </ephtml> for <emph>τ</emph>, which are consistent with estimates of <emph>θ</emph> and <emph>τ</emph> empirically observed in the CDSR, meaning that both indicate odds ratios within reasonable ranges, and heterogeneity mostly below 1.0.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-fig-0001.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-fig-0001.jpg" title="The distribution of numbers of studies included in each meta‐analysis obtained from the Cochrane Database of Systematic Reviews (CDSR). The category labelled 19+ corresponds to meta analyses of size 19 or larger [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-fig-0002.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-fig-0002.jpg" title="The distribution of the estimates of the mean treatment effect parameter θ A, and the distribution of the estimates of the (nonzero) heterogeneity standard deviation parameter ⊤ B, obtained from the reanalysis of meta‐analysis datasets in Cochrane Database of Systematic Reviews (CDSR) when the binomial‐normal hierarchical model (BNHM) via maximum likelihood estimate (MLE) is used for estimation. In A, two red lines (−1.94 and 2.06) show the 2.5% and 97.5% quantiles of the θ estimates, respectively. In B, the solid red line (1.05) and the dashed red line (1.51) indicate the 95% quantiles of the ⊤ estimates including zero‐estimates and excluding zero‐estimates, respectively. The fraction of zero‐estimates of ⊤ is 63% [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <hd id="AN0141251721-10">4 IMPLEMENTATION OF THE PROPOSED PROCEDURE IN R USING STAN</hd> <p>The Bayesian implementation of the BNHM can be fitted with the probabilistic programming language <bold>Stan</bold>,[<reflink idref="bib42" id="ref51">42</reflink>] which employs a modern Markov chain Monte Carlo (MCMC) algorithm, namely, Hamiltonian Monte Carlo with the No‐U‐Turn Sampler.[<reflink idref="bib43" id="ref52">43</reflink>] It is known that the parametrization of the model can affect the performance of an MCMC algorithm. In the presence of sparse data such as in the meta‐analysis of few studies involving rare events, Betancourt et al[<reflink idref="bib44" id="ref53">44</reflink>] showed that centered parametrization of a hierarchical model (such as the BNHM) brings computational issues compared with a noncentered parametrization. Thus, we use the noncentered reparametrized version of the BNHM for our implementations. Specifically, applying both location and scale reparametrization, becomes <emph>μ</emph><subs><emph>i</emph></subs> + <emph>θ</emph><emph>x</emph><subs><emph>ij</emph> + </subs><emph>u</emph><subs><emph>i</emph></subs> τ<emph>x</emph><subs><emph>ij</emph></subs> where <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0020" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>u</mi><mi>i</mi></msub><mo>∼</mo><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>)</mo></math> </ephtml> and <emph>x</emph><subs><emph>ij</emph></subs>=+0.5 (experimental) or <emph>x</emph><subs><emph>ij</emph></subs>=−0.5 (control). (Correction added on 06 January 2020, after first online publication: The preceding equation has been updated from "<emph>μ</emph><subs><emph>i</emph></subs> + <emph>θ</emph><subs><emph>i</emph></subs> <emph>x</emph><subs><emph>ij</emph></subs> + <emph>u</emph><subs><emph>i</emph></subs> τ<sups>2</sups>" to "<emph>μ</emph><subs><emph>i</emph></subs> + <emph>θ</emph> <emph>x</emph><subs><emph>ij</emph></subs> + <emph>u</emph><subs><emph>i</emph></subs> τ <emph>x</emph><subs><emph>ij</emph></subs>".)</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-gra-0001.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-gra-0001.jpg" title="." /> </p> <p></p> <p>For practical applications, learning <bold>Stan</bold>'s syntax and the required knowledge of available features in <bold>Stan</bold> might present a hurdle preventing application of <bold>Stan</bold>. To this end, we developed a new <bold>R</bold> package MetaStan which is a purpose‐built package defined on top of Rstan, the <bold>R</bold> interface for <bold>Stan</bold>. Our package MetaStan(https://CRAN.R-project.org/package=MetaStan) includes the precompiled <bold>Stan</bold> model of the BNHM, which eliminates the compilation time and the need of learning <bold>Stan</bold>'s syntax. The <bold>Stan</bold> code for the BNHM is shown in Listing 1. MetaStan includes different options for WIPs of the model parameters of the BNHM. MetaStan syntax is similar to the syntax of the popular meta‐analysis package metafor[<reflink idref="bib27" id="ref54">27</reflink>] so that it should be easy for a metafor user to utilize our package. The syntax of MetaStan is displayed for the pediatric transplantation example in Section , and in Appendix A, we show how to install and use MetaStan.</p> <hd id="AN0141251721-12">5 SIMULATION STUDY</hd> <p>In order to assess the long‐run properties of the proposed approach and compare it with some alternatives, we conducted a simulation study.</p> <hd id="AN0141251721-13">5.1 Simulation setup</hd> <p>The simulation scenarios are similar to those considered by Friede et al,[<reflink idref="bib24" id="ref55">24</reflink>] but with the important difference that we focus on rare events. The datasets are generated under the BNHM, more specifically. Numbers of studies (<emph>k</emph>∈{2,3,5}) and true treatment effects (<emph>θ</emph>={−5,−4,−3,−2,−1,−0.5,0,0.5,1,2,3,4,5}) are varied, resulting in a total of 39 simulation scenarios. To reflect the rare‐event cases, true baseline risks on the probability scale are taken uniformly between 0.005 and 0.05. Following Kuss,[<reflink idref="bib3" id="ref56">3</reflink>] a log‐normal distribution is fitted to the sample sizes obtained from the CDSR data, resulting in a log‐normal distribution with parameters <emph>μ</emph>=5 and <emph>σ</emph>=1. Hence, sample sizes are generated from <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0021" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">L</mi><mi mathvariant="script">N</mi><mo>(</mo><mn>5</mn><mo>,</mo><mn>1</mn><mo>)</mo></math> </ephtml> , the minimum sample size is restricted to two patients (values below 2 are rounded up to 2), and at least one patient in each treatment arm is assumed. The degree of heterogeneity (<emph>τ</emph>) is taken as <emph>τ</emph>=0.28 (moderate heterogeneity), which is the median value of the predictive distribution for between‐study heterogeneity in a meta‐analysis in a general setting as estimated by Turner et al.[<reflink idref="bib40" id="ref57">40</reflink>] According to a binomial probability of 0.5, patients were allocated to the treatment groups, thus mimicking randomization. The simulations were carried out with 10 000 replications per scenario. The data sparsity is reflected in the average fractions of single‐zero or double‐zero studies in a simulated meta‐analysis dataset, which are shown in Figure A. Notice that the fractions of the single‐zero and double‐zero studies are the highest when true treatment effect is −5, and they are decreasing with the increase of the treatment effect.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-fig-0003.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-fig-0003.jpg" title="The average fraction of single‐zero or double‐zero studies in a simulated meta‐analysis dataset A, and the fraction of the estimation failure for maximum likelihood estimate (MLE) and Mantel‐Haenszel (MH) with different numbers of studies k used in the simulations (B and C) are shown [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <p>The proposed approach (BNHM using a WIP, that is <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0022" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>,</mo><mn>2</mn><mo>.</mo><mn>8</mn><msup><mn>2</mn><mn>2</mn></msup><mo>)</mo></math> </ephtml> , for <emph>θ</emph>: <emph> WIP </emph>) and four comparators are included in the analysis, namely, BNHM using a vague prior ( <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0023" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>,</mo><mn>10</mn><msup><mn>0</mn><mn>2</mn></msup><mo>)</mo></math> </ephtml> ) for <emph>θ</emph> (<emph> Vague </emph>), BNHM using MLE (<emph> MLE </emph>), the Mantel‐Haenszel (<emph> MH </emph>) method[<reflink idref="bib5" id="ref58">5</reflink>] and a Bayesian implementation of the beta‐binomial model (<emph> BBM </emph>).[<reflink idref="bib3" id="ref59">3</reflink>] It is important to note the differences of the MH and BBM from the BNHM methods. MH is a fixed‐effect meta‐analysis method, and BBM has a different underlying data generating process than the BNHM. For both Vague and WIP approaches, the prior for <emph>τ</emph> and <emph>μ</emph> are taken as <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0024" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">H</mi><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>.</mo><mn>5</mn><mo>)</mo></math> </ephtml> , and <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0025" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>,</mo><mn>1</mn><msup><mn>0</mn><mn>2</mn></msup><mo>)</mo></math> </ephtml> , respectively. The MH estimator of the treatment effect parameter is given by</p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0026" xmlns="http://www.w3.org/1998/Math/MathML"><mrow /></math> </ephtml> </p> <p> <ephtml> <math display="block" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0027" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mover accent="true"><mi>θ</mi><mo>^</mo></mover></mrow><mtext>MH</mtext></msub><mo>=</mo><mfrac><mrow><msubsup><mrow><mo movablelimits="false">∑</mo></mrow><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>k</mi></mrow></msubsup><mfrac><mrow><msub><mi>r</mi><mrow><mi>i</mi><mn>1</mn></mrow></msub><mo>(</mo><msub><mi>n</mi><mrow><mi>i</mi><mn>0</mn></mrow></msub><mo>−</mo><msub><mi>r</mi><mrow><mi>i</mi><mn>0</mn></mrow></msub><mo>)</mo></mrow><mrow><msub><mi>n</mi><mi>i</mi></msub></mrow></mfrac></mrow><mrow><msubsup><mrow><mo movablelimits="false">∑</mo></mrow><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>k</mi></mrow></msubsup><mfrac><mrow><msub><mi>r</mi><mrow><mi>i</mi><mn>0</mn></mrow></msub><mo>(</mo><msub><mi>n</mi><mrow><mi>i</mi><mn>1</mn></mrow></msub><mo>−</mo><msub><mi>r</mi><mrow><mi>i</mi><mn>1</mn></mrow></msub><mo>)</mo></mrow><mrow><msub><mi>n</mi><mi>i</mi></msub></mrow></mfrac></mrow></mfrac><mo>,</mo></mrow></math> </ephtml> </p> <p>where <emph>n</emph><subs><emph>i</emph></subs>=<emph>n</emph><subs><emph>i</emph>0</subs>+<emph>n</emph><subs><emph>i</emph>1</subs>. In the BBM, the event counts <emph>r</emph><subs><emph>ij</emph></subs> are modeled using a binomial distribution, <emph>r</emph><subs><emph>ij</emph></subs>∼Bin(<emph>π</emph><subs><emph>ij</emph></subs>,<emph>n</emph><subs><emph>ij</emph></subs>), as in the BNHM. The probabilities of event are assumed to be beta distributed: <emph>π</emph><subs><emph>ij</emph></subs>∼Beta(<emph>α</emph><subs><emph>j</emph></subs>,<emph>β</emph><subs><emph>j</emph></subs>) where both arms share the same correlation parameter <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0028" xmlns="http://www.w3.org/1998/Math/MathML"><mi>ρ</mi><mo>=</mo><mfrac><mn>1</mn><mrow><msub><mi>α</mi><mn>0</mn></msub><mo>+</mo><msub><mi>β</mi><mn>0</mn></msub><mo>+</mo><mn>1</mn></mrow></mfrac><mo>=</mo><mfrac><mn>1</mn><mrow><msub><mi>α</mi><mn>1</mn></msub><mo>+</mo><msub><mi>β</mi><mn>1</mn></msub><mo>+</mo><mn>1</mn></mrow></mfrac></math> </ephtml> , implying <emph>α</emph><subs>0</subs>+<emph>β</emph><subs>0</subs>=<emph>α</emph><subs>1</subs>+<emph>β</emph><subs>1</subs>. It is common to reparametrize the model using mean parameters Φ<subs><emph>j</emph></subs> such that <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0029" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi mathvariant="normal">Φ</mi><mi>j</mi></msub><mo>=</mo><mfrac><mrow><msub><mi>α</mi><mi>j</mi></msub></mrow><mrow><msub><mi>α</mi><mi>j</mi></msub><mo>+</mo><msub><mi>β</mi><mi>j</mi></msub></mrow></mfrac></math> </ephtml> . Finally, the linear predictor can be written as logit(Φ<subs><emph>j</emph></subs>)=<emph>μ</emph>+<emph>θ</emph> <emph>x</emph><subs><emph>j</emph></subs>  where <emph>θ</emph> is the parameter for the treatment effect, and <emph>x</emph><subs><emph>j</emph></subs> is a treatment indicator, 1 = experimental (<emph>j</emph> = 1) and 0 = control (<emph>j</emph> = 0). Vague priors are chosen for all parameters, namely, uniform priors across the interval [0,1] for all three parameters: Φ<subs>0</subs>, Φ<subs>1</subs>, and <emph>ρ</emph>.</p> <p>Three MCMC chains were run in parallel for a total of 2000 iterations including 1000 iterations of burn‐in. These values are tested in some replications; convergence diagnostics are assessed and chosen accordingly. All chains were assumed to have reached convergence (no estimation failure). We used the package lme4 for the MLE (using the adaptive Gauss‐Hermite approximation to the maximum log‐likelihood) and metafor for the MH (without using any continuity corrections) whereas the Vague, WIP, and BBM methods were fitted with our <bold>MetaStan</bold> package. Note that we use highest density intervals (HDI), which are the shortest credible intervals, as opposed to the commonly used equal‐tailed credible intervals. The HDI were obtained using the HDInterval[<reflink idref="bib45" id="ref60">45</reflink>] package. All computations were performed using <bold>R</bold>.[<reflink idref="bib46" id="ref61">46</reflink>] The code for the computations for all methods used in the simulations is provided in Appendices A to D.</p> <hd id="AN0141251721-15">5.2 Simulation results</hd> <p>For the MLE and the MH, the fractions of estimation failures are shown in Figure B and  C. Estimation failure occurred for the MLE when the Gauss‐Hermite approximation does not converge to the maximum log‐likelihood, or when the MH estimator is not defined. The MLE and MH methods show very similar behavior of the estimation failure. Estimation failure is closely related to the fraction of meta‐analysis datasets including single‐zero or double‐zero studies in the dataset, which can be seen by comparing Figure A and B,C. This is because when the data are highly sparse, estimation becomes more challenging for both MLE and MH. As a performance measure, we use the bias ( <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0030" xmlns="http://www.w3.org/1998/Math/MathML"><mfrac><mn>1</mn><mi>N</mi></mfrac><msubsup><mrow><mo>∑</mo></mrow><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>N</mi></mrow></msubsup><mo>(</mo><mover accent="true"><mi>θ</mi><mo>^</mo></mover><mo>−</mo><mi>θ</mi><mo>)</mo></math> </ephtml> ) based on the MLE, the MH estimator, and posterior medians. The direction of the bias is also important, since depending on the nature of the outcome (safety or efficacy), a positive or a negative bias may be considered <emph> conservative </emph>. Moreover, the coverage probability and the mean length of interval estimates for <emph>θ</emph> are reported. The coverage probability of  95% for interval estimates and shorter interval estimates are desirable.</p> <p>The bias of posterior medians from the Vague, the WIP and the BBM, and for MLEs from the MLE and for the MH estimator from the MH across scenarios is displayed in the first row of Figure. Note that failed runs were excluded from the calculation of performance measures that is relevant only for MLE and MH. The MLE shows unacceptably high bias for the scenarios with <emph>θ</emph> ≤ 0, corresponding to the scenarios in which the fraction of zero studies is also very high. On the other hand, the MH estimator clearly outperforms the MLE and exhibit bias very close to the WIP. The WIP displays somewhat positive bias whereas the Vague shows negative bias for the scenarios with <emph>θ</emph> ≤ 0. This behavior of WIP is expected, since the WIP shrinks the posterior towards zero. For safety analyses, a positive bias commonly means a more conservative behavior and may hence be considered less harmful than a negative bias. It is important to note that the results of the bias behave similar to the fraction of zero studies and the fraction of estimation failure of the MLE, meaning that the bias is higher in scenarios with more sparse data. Since the Vague approach uses a vague prior on <emph>θ</emph>, one might expect a somewhat similar behavior of bias from the Vague and the MLE approaches. However, the fact that the Vague approach includes a WIP for <emph>τ</emph> and that estimation is based on integration rather than maximization may be explanations of the better performance of the Vague method in comparison with the MLE. The WIP and the MH outperforms the BBM in terms of bias across all scenarios. Performance in terms of bias is improving for all methods when the number of studies <emph>k</emph> is increasing. For Figures and , the curves are not symmetric around zero. This asymmetry is due to the fact that while the true treatment effect (log‐OR) is varied between −5 and +5, the true baseline risk (probability) is drawn uniformly between 0.005 and 0.05 in the simulations.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-fig-0004.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-fig-0004.jpg" title="The bias for the mean treatment effect θ, coverage probabilities, and log mean length of the interval estimates for θ obtained by five methods (beta‐binomial model [BBM], Mantel‐Haenszel [MH], maximum likelihood estimate [MLE], Vague, and weakly informative prior [WIP]) are shown [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <p>Figure also shows coverage probabilities and log mean lengths for 95% HDI obtained by the Vague, the WIP, the BBM, and for 95% Wald confidence intervals (CIs) obtained by the MLE and the MH. The CI and HDI obtained by the MH and the BBM show unacceptably low coverage especially for <emph>θ</emph><−2. However, the undercoverage of the BBM and somewhat relative poor performance in terms of bias may stem from the fact that data are generated under the BNHM. Also, the CI obtained by the MLE displays low coverage especially for <emph>k</emph>=5. We will revisit the coverage of the MLE in the discussion. The WIP method shows higher coverage than nominal level across all different true treatment effects except for <emph>θ</emph>=−5. On the other hand, the HDI obtained by WIP are shorter in comparison with HDI obtained by the Vague and CI obtained by the MLE approaches.</p> <p>Lastly, the bias for the heterogeneity parameter <emph>τ</emph> obtained by three methods (the MLE, the Vague, and the WIP) are demonstrated in Figure. For Bayesian methods, posterior medians are used as the point estimates. Recall that the prior used for <emph>τ</emph> both in the Vague and the WIP is weakly informative ( <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0031" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">H</mi><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>.</mo><mn>5</mn><mo>)</mo></math> </ephtml> ). The MLE underestimates the true heterogeneity, whereas the Vague and the WIP methods slightly overestimate it. The Vague and the WIP produce very similar bias. These observations are in alignment with the conclusions made by Friede et al.[<reflink idref="bib24" id="ref62">24</reflink>]</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-fig-0005.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-fig-0005.jpg" title="The bias for the heterogeneity parameter ⊤ obtained by three methods (maximum likelihood estimate [MLE], Vague, and weakly informative prior [WIP]) is shown. True heterogeneity standard deviation is assumed to be ⊤=0.28 [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <hd id="AN0141251721-18">6 EXAMPLE REVISITED</hd> <p>Returning to the dataset described in Section , we consider the data on death and PTLD outcomes shown in Table. The observed log‐odds ratios are displayed in Figures and . To be able to visualize the observed log‐odds ratios when there is a single‐zero or double‐zero study, a continuity correction of 0.5 is added to all cells of the single‐zero or double‐zero study's contingency table. The wide CI for observed log‐odds ratios reflect the rather small sample sizes in the datasets. Furthermore, the variability in the point estimates may be reflected upon to assess the degree of heterogeneity between trial estimates.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-fig-0006.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-fig-0006.jpg" title="The motivating pediatric transplantation application when the outcome is death: Top panel displays the observed log‐odds ratios (computed using a continuity correction in case of zero counts). The bottom panel shows mean treatment effect estimates of θ obtained by beta‐binomial model (BBM), Mantel‐Haenszel (MH), maximum likelihood estimate (MLE), Vague, and weakly informative prior (WIP). Heterogeneity parameter estimates ⊤ are also given on the left [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-fig-0007.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-fig-0007.jpg" title="The motivating pediatric transplantation application when the outcome is posttransplant lymphoproliferative disease (PTLD): Top panel displays the observed log‐odds ratios (computed using a continuity correction in case of zero counts). The bottom panel shows mean treatment effect estimates of θ obtained by beta‐binomial model (BBM), Mantel‐Haenszel (MH), maximum likelihood estimate (MLE), Vague, and weakly informative prior (WIP). Heterogeneity parameter estimates ⊤ are also given on the left [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <p>We analyze the datasets using the five methods investigated in the simulation studies, namely, the Vague, WIP, MLE, MH, and BBM approaches. The code to implement the MLE and the MH are given in Appendix B. Recall that the only difference between Vague and WIP is the prior used for the treatment effect parameter <emph>θ</emph> in the model, namely, <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0032" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>,</mo><mn>10</mn><msup><mn>0</mn><mn>2</mn></msup><mo>)</mo></math> </ephtml> for the former and <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0033" xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="script">N</mi><mo>(</mo><mn>0</mn><mo>,</mo><mn>2</mn><mo>.</mo><mn>8</mn><msup><mn>2</mn><mn>2</mn></msup><mo>)</mo></math> </ephtml> for the latter. WIP can be implemented in a routine data analysis using our MetaStan package as follows:</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-gra-0002.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-gra-0002.jpg" title="." /> </p> <p></p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-gra-0003.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-gra-0003.jpg" title="." /> </p> <p></p> <p>The argument delta corresponds to <emph>δ</emph> from and thus is used to calculate the WIP for <emph>θ</emph>. Alternatively, one can directly specify the prior parameters for <emph>θ</emph>, in our case, equivalently, we can have theta_prior = c(0, 2.82). The Vague method is simply implemented by omitting the argument delta and specifying theta_prior = c(0, 100). The BBM is also implemented in MetaStan, and the required syntax is shown in Appendix C. To check MCMC convergence, we use the Gelman‐Rubin statistics and traceplots. For the WIP approach, the corresponding traceplots are shown in Figures A1 and A2 for death and PTLD outcomes, respectively. There was no divergence reported for both datasets. The MLE fit and the MH estimation for the dataset where death is the outcome does not cause any warning from lme4 and metafor, respectively. For the PTLD outcome, lme4 gives a warning suggesting that the estimates may not be reliable. Nevertheless, it produces the MLE estimate and CI for treatment effect parameter, and we report them. For PTLD outcomes, when computing the MH estimator, metafor gives a warning due to double‐zero studies (double‐zero studies are removed from the analysis by default) but still returns an estimate. Note that both MLE and MH ignore the double‐zero study (Schuller et al[<reflink idref="bib30" id="ref63">30</reflink>]); hence, the analyses are based on two studies only.</p> <p>The results for the death and PTLD outcomes from the five methods are shown in Figures and , respectively. For MLE and MH, the estimates and 95% CI are given. For Vague, WIP, and BBM, posterior medians and 95% HDI are shown. Both for PTLD and death outcomes, apart from the BBM, the point estimates of <emph>θ</emph> from the four methods look quite similar. The differing behavior of the BBM was also observed in the simulations. The PTLD data are similar to the scenarios when the number of studies is three, and the true treatment effect is in the range from 0 to 1. Negative bias obtained by the BBM can be seen in Figure (in the corresponding scenario). The death data are similar to the scenarios when number of studies is five (since it is not highly sparse), and true treatment effect is in the range from −1 to 0. Here, positive bias obtained by the BBM can be seen in Figure (in the corresponding scenario). Furthermore, the point estimates obtained by the WIP and the MH are very close as in the simulations. MLE gives shorter interval estimates compared with Bayesian alternatives, this is (partly) because <emph>τ</emph> was estimated as 0. In the original paper, Crins et al[<reflink idref="bib26" id="ref64">26</reflink>] fitted a normal‐normal hierarchical model using REML,[<reflink idref="bib27" id="ref65">27</reflink>] and the risk ratio was used as the measure of the treatment effect. They concluded that treatment IL‐2RA failed to show statistically significant result for reducing death. We obtained similar point estimates with somewhat wider interval estimates to Crins et al,[<reflink idref="bib26" id="ref66">26</reflink>] specifically their risk ratio estimate was 0.61 (CI, 0.27‐1.37), and we obtained the odds ratio estimate 0.58 (HDI 0.20‐1.49) using the WIP method. Concerning the PTLD, the risk ratio was estimated as 1.60 (CI, 0.20‐12.67) by Crins et al,[<reflink idref="bib26" id="ref67">26</reflink>] the odds ratio is estimated 1.98 (HDI 0.18‐25.18) using the WIP method. The wider interval estimates obtained by WIP may stem from the fact that the uncertainty in <emph>τ</emph> is taken into account.</p> <p>The estimates of the between‐trial heterogeneity <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0034" xmlns="http://www.w3.org/1998/Math/MathML"><mover accent="true"><mi>τ</mi><mo>^</mo></mover></math> </ephtml> are also included in the figure, which are only available for the Vague, WIP, and MLE. Considering death as outcomes, the heterogeneity parameter <emph>τ</emph> is estimated 0.29, 0.29, and 0.00 using WIP, Vague, and MLE, respectively. Similarly for the PTLD outcomes, for <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0035" xmlns="http://www.w3.org/1998/Math/MathML"><mover accent="true"><mi>τ</mi><mo>^</mo></mover></math> </ephtml> , we obtained 0.33, 0.33, and 0.00 using WIP, Vague, and MLE, respectively. The heterogeneity parameter of the BBM <emph>ρ</emph> is estimated as 0.34 and 0.03 for PTLD and death outcomes, respectively. Moreover, Crins et al[<reflink idref="bib26" id="ref68">26</reflink>] concluded that there is no evidence for heterogeneity between trials using using Cochrane's Q test for both death and PTLD outcomes. Since the prior used for <emph>τ</emph> is the same for WIP and Vague, similar heterogeneity estimates are expected. The similar <emph>τ</emph> estimates by WIP and Vague were also observed in the simulations (Figure). On the other hand, the MLE estimate ( <ephtml> <math display="inline" altimg="urn:x-wiley:jrsm:media:jrsm1370:jrsm1370-math-0036" xmlns="http://www.w3.org/1998/Math/MathML"><mover accent="true"><mi>τ</mi><mo>^</mo></mover><mo>=</mo><mn>0</mn><mo>.</mo><mn>00</mn></math> </ephtml> ) is most probably underestimating the actual amount of heterogeneity. The underestimation of <emph>τ</emph> by MLE and slightly lower bias of the WIP compared with the Vague was observed in the simulations (Figure).</p> <hd id="AN0141251721-23">7 CONCLUSIONS AND DISCUSSION</hd> <p>An assumption of the homogeneity is  often considered unrealistic for meta‐analyses in biomedical sciences; hence, random‐effects meta‐analysis models are suggested.[<reflink idref="bib6" id="ref69">6</reflink>] Furthermore, as can be seen in the CDSR, a substantial fraction of published meta‐analyses is based on few studies only. On the other hand, fitting a random‐effects models based on only few studies often poses problems for inference, as certain asymptotics cannot be relied upon.[<reflink idref="bib47" id="ref70">47</reflink>] Additional issues arise for binary outcomes when only few or no events are observed in some of the studies or study arms. To deal with such data sparsity in the meta‐analysis, we have proposed the use of WIPs for the treatment effect parameter <emph>θ</emph> in a BNHM. We demonstrated how a normal WIP for <emph>θ</emph> can be derived by considering an a priori interval for the treatment effect on a log‐odds ratio scale. Also, the empirical evidence obtained from 37 773 meta‐analyses with binomial outcomes from the CDSR supports the proposed WIP. In simulation studies, the suggested method displays lower bias for <emph>θ</emph> and substantially shorter interval estimates for <emph>θ</emph> with somewhat higher coverage than nominal level in comparison to alternative methods.</p> <p>The use of a Bayesian approach exhibits analogy of some degree to the use of continuity corrections. While continuity corrections might to some extent be perceived as ad hoc makeshift fixes, they have also quite doubtlessly proven very useful in practice. A Bayesian approach tackles the problem from a very different angle, but it is not so surprising that the resulting procedure again exhibits some similarity to continuity corrections. The relation to current common practice may in fact also be seen as somewhat comforting. Use of an (informative) prior within a Bayesian analysis on the other hand is not a desperate measure; it is rather an integral part of a coherent model specification that may also be subjected to checks of plausibility and operating characteristics; this is what we have tried to demonstrate in the present paper.</p> <p>The simulation results displayed in Figure are somewhat in contrast to the results given by Friede et al,[<reflink idref="bib24" id="ref71">24</reflink>] who observed lower coverage than nominal level of MLE methods in a similar setting, but not based on rare events. We also investigated a scenario closer to their setup by considering higher baseline risks between 0.05 and 0.20. The results are shown in Figure D1,  and indeed, here, the MLE method exhibits lower coverage than nominal level, as reported by Chung et al[<reflink idref="bib22" id="ref72">22</reflink>] and Friede et al.[<reflink idref="bib24" id="ref73">24</reflink>] The high bias and too wide interval estimates obtained by the Vague and the MLE are still present, but not as high as in the results of the simulations in which true baseline risks are lower.</p> <p>Jackson et al[<reflink idref="bib11" id="ref74">11</reflink>] investigated seven random‐effects meta‐analysis models including the BNHM which we consider in this paper (<emph>model 4 </emph> in Jackson et al[<reflink idref="bib11" id="ref75">11</reflink>]) and another parametrization of the BNHM (<emph>model 2 </emph> in Jackson et al[<reflink idref="bib11" id="ref76">11</reflink>]). The only difference in the specification between the two models is that in their Model 2, the treatment indicator <emph>x</emph><subs><emph>ik</emph></subs> of is +1 for the experimental arm, and 0 for the control arm. Note that commonly used network meta‐analysis models, for example,[<reflink idref="bib48" id="ref77">48</reflink>] are generalizations of Model 2 in Jackson et al.[<reflink idref="bib11" id="ref78">11</reflink>] As reported by Jackson et al,[<reflink idref="bib11" id="ref79">11</reflink>] we also observe the underestimation of the heterogeneity parameter <emph>τ</emph> and hence decided to only consider their model 4. On the other hand, it is important to note that the usage of a WIP for <emph>θ</emph> also improves the performance in model 2, as we have seen for the model 4.</p> <p>This investigation has some limitations. One crucial limitation is that we only considered the BNHM as a data‐generating process in our simulation study. Hence, we did not investigate the robustness of the BNHM under model misspecification. Also, the design of the simulation study constitutes a model misspecification problem for the MH method, which is a fixed‐effect model, and for the BBM, which assumes a different underlying data‐generating process. Moreover, we did not consider other parametrizations of the BNHM as described, eg, in Jackson et al.[<reflink idref="bib11" id="ref80">11</reflink>] Lastly, one may find it too restrictive to have a normal prior for <emph>θ</emph> as we have in our proposed model, it may be worth exploring alternatives like Cauchy or log‐<emph>F</emph> distributions[[<reflink idref="bib17" id="ref81">17</reflink>], [<reflink idref="bib20" id="ref82">20</reflink>]] for penalization.</p> <p>The proposed approach is not restricted to the BNHM; similar approaches may analogously be defined in other models, eg, a Poisson‐normal hierarchical model. However, a crucial point is that the treatment effect parameter is explicitly parameterized in the model, so that it can directly be <emph> penalized </emph> via the prior specification. Hence, so‐called contrast‐based models[<reflink idref="bib25" id="ref83">25</reflink>] (in which relative treatment effects are assumed to be exchangeable across trials) are suitable for this purpose unlike arm‐based models. Note that this is also related to the inclusion of baseline risks as fixed effects with vague priors. This was on purpose as we consider this closest to the idea of stratifying the analyses by study, a common feature of meta‐analyses regardless of fixed or random‐effects. Furthermore, the contrast‐based models such as the BNHM preserve the randomization, in contrast to the arm‐based models as explained in Dias and Ades.[<reflink idref="bib25" id="ref84">25</reflink>]</p> <p>The BNHM can be extended to a network meta‐analysis model,[<reflink idref="bib49" id="ref85">49</reflink>] which is desirable if there are multiple treatments, and/or multiarm trials in the dataset. Even if the dataset in a network meta‐analysis consists of many studies overall, some of the treatment effects may still be informed by few studies only. Thus, the use of WIPs for treatment effect parameters in the context of network meta‐analysis with rare events can be very helpful. Different distributions as WIP for <emph>θ</emph>, different parametrizations of BNHM, or different data models can be implemented in <bold>Stan</bold> or MCMC methods in general. Although, currently, our package MetaStan is restricted to use a BNHM and BBM for pairwise meta‐analysis, we will consider to extend it to conduct meta‐analysis and network meta‐analysis with flexible data model and prior options in the future.</p> <hd id="AN0141251721-24">ACKNOWLEDGMENT</hd> <p>We thank Leonhard Held who contributed valuable comments and pointed us to several important references.</p> <hd id="AN0141251721-25">HIGHLIGHTS</hd> <p> <bold>What is already known</bold>: Standard random‐effects meta‐analysis methods are not suitable for meta‐analysis of few studies with rare events.</p> <p> <bold>What is new</bold>: To deal with data sparsity present in the random‐effects meta‐analysis of few studies with rare events, we suggest the use of weakly informative priors as penalization for the treatment effect parameter.</p> <p> <bold>Potential impact for RSM readers outside the authors' field</bold>: To make it more accessible to meta‐analysts, a publicly available <bold>R</bold> package, MetaStan, is developed for fitting Bayesian meta‐analysis models using weakly informative priors.</p> <hd id="AN0141251721-26">CONFLICT OF INTEREST</hd> <p>The author reported no conflict of interest.</p> <p>A APPENDIX HOW TO USE THE METASTAN R PACKAGE</p> <p>The stable version of <bold>MetaStan</bold> is available on CRAN (https://CRAN.R-project.org/package=MetaStan) and can be installed as follows:</p> <p> The example described in the text (Crins dataset) is available in the package, and it can be loaded as follows:</p> <p>Additional information can be obtained by typing ?dat.Crins2014 (for any dataset and function in the package).</p> <p>meta_stan is the main fitting function of this package. The main computations are executed via the <bold>rstan</bold> package's sampling function. We can fit the binomial‐normal hierarchical using a WIP for treatment effect as follows:</p> <p>GRAPH</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-gra-0007.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-gra-0007.jpg" title="." /> </p> <p></p> <p> Convergence diagnostics and the results can be very conveniently obtained using the <bold>shinystan</bold> package as follows:</p> <p> Traceplots for the estimated parameters <emph>θ</emph> and <emph>τ</emph> including burn‐in are shown in Figures A1 and A2 for death and PTLD outcomes, respectively.</p> <p>GRAPH: A1 Traceplots for the estimated parameters θ and  ⊤  including burn‐in for death outcomes [Colour figure can be viewed at wileyonlinelibrary.com]</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-fig-0008.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-fig-0008.jpg" title="." /> </p> <p></p> <p>GRAPH: A2 Traceplots for the estimated parameters θ and⊤including burn‐in for posttransplant lymphoproliferative disease (PTLD) outcomes [Colour figure can be viewed at wileyonlinelibrary.com]</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-fig-0009.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-fig-0009.jpg" title="." /> </p> <p></p> <p>Lastly, the posterior summary statistics can be obtained using the following command:</p> <p>B APPENDIX R CODE TO IMPLEMENT BNHM USING THE MLE AND THE MH METHODS</p> <p>Firstly, the BNHM using the MLE:</p> <p> Secondly, the MH method:</p> <p>C APPENDIX R CODE TO IMPLEMENT THE BBM METHOD</p> <p>D APPENDIX ADDITIONAL SIMULATION RESULTS</p> <p>We also conducted simulations using the same settings as described in Section under BNHM, but using higher baseline risk probabilities, specifically, baseline risks (<emph>μ</emph><subs><emph>i</emph></subs>) are uniformly taken between 0.05 and 0.2. Results are illustrated in Figure D1 (analogous to Figure).</p> <p>GRAPH: D1 Simulations with high baseline risks: The bias for the mean treatment effect θ, coverage probabilities, and log mean length of the interval estimates for θ obtained by three methods (maximum likelihood estimate [MLE], Vague, and weakly informative prior [WIP]) are shown [Colour figure can be viewed at wileyonlinelibrary.com]</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01jan20/jrsm1370-fig-0010.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1370-fig-0010.jpg" title="." /> </p> <p></p> <ref id="AN0141251721-31"> <title> REFERENCES </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Chichester: Wiley; 2008.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref2" type="bt">2</bibl> <bibtext> Friedrich JO, Adhikari NKJ, Beyene J. Inclusion of zero total event trials in meta‐analyses maintains analytic consistency and incorporates all available data. BMC Med Res Methodol. 2007 ; 7 (1): 5. https://doi.org/10.1186/1471-2288-7-5</bibtext> </blist> <blist> <bibl id="bib3" idref="ref3" type="bt">3</bibl> <bibtext> Kuss O. Statistical methods for meta‐analyses including information from studies without any events–add nothing to nothing and succeed nevertheless. Stat Med. 2015 ; 34 (7): 1097 ‐ 1116.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref4" type="bt">4</bibl> <bibtext> Yusuf S, Peto R, Lewis J, Collins R, Sleight P. Beta blockade during and after myocardial infarction: an overview of the randomized trials. Prog Cardiovasc Dis. 1985 ; 27 (5): 335 ‐ 371.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref5" type="bt">5</bibl> <bibtext> Mantel N, Haenszel W. Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst Monogr. 1959 ; 22 (4): 719 ‐ 748. https://doi.org/10.1093/jnci/22.4.719</bibtext> </blist> <blist> <bibl id="bib6" idref="ref6" type="bt">6</bibl> <bibtext> Higgins JPT, Thompson SG, Spiegelhalter DJ. A re‐evaluation of random‐effects meta‐analysis. J R Stat Soc Ser A Stat Soc. 2009 ; 172 (1): 137 ‐ 159. https://doi.org/10.1111/j.1467-985X.2008.00552.x</bibtext> </blist> <blist> <bibl id="bib7" type="bt">7</bibl> <bibtext> Turner RM, Jackson D, Wei Y, Thompson SG, Higgins JPT. Predictive distributions for between‐study heterogeneity and simple methods for their application in Bayesian meta‐analysis. Stat Med. 2015 ; 34 (6): 984 ‐ 998.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref45" type="bt">8</bibl> <bibtext> Kontopantelis E, Springate DA, Reeves D. A re‐analysis of the cochrane library data: the dangers of unobserved heterogeneity in meta‐analyses. PLoS One. 2013 ; 8 (7): 1 ‐ 14.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref7" type="bt">9</bibl> <bibtext> Hedges LV, Olkin I. Random effects models for effect sizes. Statistical methods for meta‐analysis ; 1985.</bibtext> </blist> <blist> <bibtext> Bradburn MJ, Deeks JJ, Berlin JA, Russell Localio A. Much ado about nothing: a comparison of the performance of meta‐analytical methods with rare events. Stat Med. 2007 ; 26 (1): 53 ‐ 77. https://doi.org/10.1002/sim.2528</bibtext> </blist> <blist> <bibtext> Jackson D, Law M, Stijnen T, Viechtbauer W, White IR. A comparison of seven random‐effects models for meta‐analyses that estimate the summary odds ratio. Stat Med. 2018 ; 37 (7): 1059 ‐ 1085.</bibtext> </blist> <blist> <bibtext> Stijnen T, Hamza TH, Özdemir P. Random effects meta‐analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data. Stat Med. 2010 ; 29 (29): 3046 ‐ 3067.</bibtext> </blist> <blist> <bibtext> Böhning D, Mylona K, Kimber A. Meta‐analysis of clinical trials with rare events. Biom J. 2015 ; 57 (4): 633 ‐ 648. https://doi.org/10.1002/bimj.201400184</bibtext> </blist> <blist> <bibtext> Cai T, Parast L, Ryan L. Meta‐analysis for rare events. Stat Med. 2010 ; 29 (20): 2078 ‐ 2089. https://doi.org/10.1002/sim.3964</bibtext> </blist> <blist> <bibtext> Smith TC, Spiegelhalter DJ, Thomas A. Bayesian approaches to random‐effects meta‐analysis: a comparative study. Stat Med. 1995 ; 14 (24): 2685 ‐ 2699. https://doi.org/10.1002/sim.4780142408</bibtext> </blist> <blist> <bibtext> Albert A, Anderson JA. On the existence of maximum likelihood estimates in logistic regression models. Biometrika. 1984 ; 71 (1): 1 ‐ 10.</bibtext> </blist> <blist> <bibtext> Greenland S, Mansournia MA. Penalization, bias reduction, and default priors in logistic and related categorical and survival regressions. Stat Med. 2015 ; 34 (23): 3133 ‐ 3143.</bibtext> </blist> <blist> <bibtext> Firth D. Bias reduction of maximum likelihood estimates. Biometrika. 1993 ; 80 (1): 27 ‐ 38.</bibtext> </blist> <blist> <bibtext> Heinze G, Schemper M. A solution to the problem of separation in logistic regression. Stat Med. 2002 ; 21 (16): 2409 ‐ 2419.</bibtext> </blist> <blist> <bibtext> Gelman A, Jakulin A, Pittau MG, Su Y. A weakly informative default prior distribution for logistic and other regression models. Ann Appl Stat. 2008 ; 2 (4): 1360 ‐ 1383.</bibtext> </blist> <blist> <bibtext> Davey J, Turner RM, Clarke MJ, Higgins JPT. Characteristics of meta‐analyses and their component studies in the Cochrane Database of Systematic Reviews: a cross‐sectional, descriptive analysis. BMC Med Res Methodol. 2011 ; 11 (1): 160.</bibtext> </blist> <blist> <bibtext> Chung Y, Rabe‐Hesketh S, Choi IH. Avoiding zero between‐study variance estimates in random‐effects meta‐analysis. Stat Med. 2013 ; 32 (23): 4071 ‐ 4089.</bibtext> </blist> <blist> <bibtext> Gelman A. Prior distributions for variance parameters in hierarchical models (comment on article by browne and draper). Bayesian Anal. 2006 ; 1 (3): 515 ‐ 534. https://doi.org/10.1214/06-BA117A</bibtext> </blist> <blist> <bibtext> Friede T, Röver C, Wandel S, Neuenschwander B. Meta‐analysis of few small studies in orphan diseases. Res Synth Methods. 2017 ; 8 (1): 79 ‐ 91. https://doi.org/10.1002/jrsm.1217</bibtext> </blist> <blist> <bibtext> Dias S, Ades AE. Absolute or relative effects? Arm‐based synthesis of trial data. Res Synth Methods. 2016 ; 7 (1): 23 ‐ 28. https://doi.org/10.1002/jrsm.1184</bibtext> </blist> <blist> <bibtext> Crins ND, Röver C, Goralczyk AD, Friede T. Interleukin‐2 receptor antagonists for pediatric liver transplant recipients: a systematic review and meta‐analysis of controlled studies. Pediatr Transplant. 2014 ; 18 (8): 839 ‐ 850. https://doi.org/10.1111/petr.12362</bibtext> </blist> <blist> <bibtext> Viechtbauer W. Conducting meta‐analyses in R with the metafor package. J Stat Softw. 2010 ; 36 (3): 1 ‐ 48. <ulink href="http://www.jstatsoft.org/v36/i03/">http://www.jstatsoft.org/v36/i03/</ulink></bibtext> </blist> <blist> <bibtext> Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta‐analyses. BMJ. 2003 ; 327 : 557 ‐ 560.</bibtext> </blist> <blist> <bibtext> Heffron TG, Pillen T, Smallwood GA, Welch D, Oakley B, Romero R. Pediatric liver transplantation with daclizumab induction therapy. Transplant. 2003 ; 75 : 2040 ‐ 2043.</bibtext> </blist> <blist> <bibtext> Schuller S, Wiederkehr JC, Coelho‐Lemos IM, Avilla SG, Schultz C. Daclizumab induction therapy associated with tacrolimus‐mmf has better outcome compared with tacrolimus‐MMF alone in pediatric living donor liver transplantation. Transplant Proc. 2005 ; 37 (2): 1151 ‐ 1152.</bibtext> </blist> <blist> <bibtext> Ganschow R, Grabhorn E, Schulz A, Hugo AV, Rogiers X, Burdelski M. Long‐term results of basiliximab induction immunosuppression in pediatric liver transplant recipients. Pediatr Transplant. 2005 ; 9 (6): 741 ‐ 745.</bibtext> </blist> <blist> <bibtext> Spada M, Petz W, Bertani A, et al. Randomized trial of basiliximab induction versus steroid therapy in pediatric liver allograft recipients under tacrolimus immunosuppression. Am J Transplant. 2006 ; 6 (8): 1913 ‐ 1921.</bibtext> </blist> <blist> <bibtext> Gras JM, Gerkens S, Beguin C, et al. Steroid‐free, tacrolimus‐basiliximab immunosuppression in pediatric liver transplantation: clinical and pharmacoeconomic study in 50 children. Liver Transpl. 2008 ; 14 (4): 469 ‐ 477.</bibtext> </blist> <blist> <bibtext> Spiegelhalter DJ, Abrams KR, Myles JP. Prior distributions. Bayesian Approaches to Clinical Trials and Health‐Care Evaluation. West Sussex : CRC Press ; 2004.</bibtext> </blist> <blist> <bibtext> Greenland S. Bayesian perspectives for epidemiological research: I. Foundations and basic methods. Int J Epidemiol. 2006 ; 35 (3): 765 ‐ 775.</bibtext> </blist> <blist> <bibtext> Kass RE, Wasserman L. A reference Bayesian test for nested hypotheses and its relationship to the schwarz criterion. J Am Stat Assoc. 1995 ; 90 (431): 928 ‐ 934.</bibtext> </blist> <blist> <bibtext> Röver C. Bayesian random‐effects meta‐analysis using the bayesmeta R package. arxiv.org e‐print archive. https://arxiv.org/abs/1711.08683 updated november 23, 2017. accessed August 30, 2018.</bibtext> </blist> <blist> <bibtext> The cochrane. cochrane database of systematic reviews. https://<ulink href="http://www.cochranelibrary.com">www.cochranelibrary.com</ulink> accessed april 15, 2018.</bibtext> </blist> <blist> <bibtext> Springate D. Cochrane_scraper v1.1.0. https://doi.org/10.5281/zenodo.10782 updated july, 2014. accessed april, 2018.</bibtext> </blist> <blist> <bibtext> Turner RM, Davey J, Clarke MJ, Thompson SG, Higgins JPT. Predicting the extent of heterogeneity in meta‐analysis, using empirical data from the cochrane Database of Systematic Reviews. Int J Epidemiol. 2012 ; 41 (3): 818 ‐ 827. https://doi.org/10.1093/ije/dys041</bibtext> </blist> <blist> <bibtext> Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed‐effects models using lme4. J Stat Softw. 2015 ; 67 (1): 1 ‐ 48.</bibtext> </blist> <blist> <bibtext> Stan Development Team. Stan modeling language user's guide and reference manual, version 2.17.0. ; 2017.</bibtext> </blist> <blist> <bibtext> Betancourt M. A conceptual introduction to Hamiltonian Monte Carlo. arxiv.org e‐print archive. https://arxiv.org/abs/1701.02434. updated july 16, 2018. accessed august 24, 2018.</bibtext> </blist> <blist> <bibtext> Betancourt M, Girolami M. Hamiltonian Monte Carlo for hierarchical models. Current Trends in Bayesian Methodology With Applications ; 2015.</bibtext> </blist> <blist> <bibtext> Meredith M, kruschke J. HDInterval: Highest (posterior) density intervals, R package version 0.2.0. https://CRAN.R-project.org/package=HDInterval ; 2018.</bibtext> </blist> <blist> <bibtext> R core team. R: A language and environment for statistical computing ; R foundation for statistical computing. https://<ulink href="http://www.R-project.org/">www.R-project.org/</ulink> Vienna, Austria. 2018.</bibtext> </blist> <blist> <bibtext> Jackson D, White IR. When should meta‐analysis avoid making hidden normality assumptions?. Biom J. 2018 ; 60 (6): 1040 ‐ 1058. https://onlinelibrary.wiley.com/doi/abs/10.1002/bimj.201800071</bibtext> </blist> <blist> <bibtext> Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical support document 2: A generalised linear modelling framework for pairwise and network meta‐analysis of randomised controlled trials. <ulink href="http://www.nicedsu.org.uk">http://www.nicedsu.org.uk</ulink> last updated September 2016; 2011.</bibtext> </blist> <blist> <bibtext> Günhan BK, Friede T, Held L. A design‐by‐treatment interaction model for network meta‐analysis and meta‐regression with integrated nested laplace approximations. Res Synth Methods. 2018 ; 9 (2): 179 ‐ 194.</bibtext> </blist> </ref> <aug> <p>By Burak Kürsad Günhan; Christian Röver and Tim Friede</p> <p>Reported by Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib10" firstref="ref8"></nolink> <nolink nlid="nl2" bibid="bib11" firstref="ref9"></nolink> <nolink nlid="nl3" bibid="bib12" firstref="ref10"></nolink> <nolink nlid="nl4" bibid="bib13" firstref="ref11"></nolink> <nolink nlid="nl5" bibid="bib14" firstref="ref12"></nolink> <nolink nlid="nl6" bibid="bib15" firstref="ref14"></nolink> <nolink nlid="nl7" bibid="bib16" firstref="ref15"></nolink> <nolink nlid="nl8" bibid="bib17" firstref="ref16"></nolink> <nolink nlid="nl9" bibid="bib20" firstref="ref18"></nolink> <nolink nlid="nl10" bibid="bib21" firstref="ref19"></nolink> <nolink nlid="nl11" bibid="bib22" firstref="ref20"></nolink> <nolink nlid="nl12" bibid="bib23" firstref="ref21"></nolink> <nolink nlid="nl13" bibid="bib24" firstref="ref22"></nolink> <nolink nlid="nl14" bibid="bib25" firstref="ref25"></nolink> <nolink nlid="nl15" bibid="bib26" firstref="ref26"></nolink> <nolink nlid="nl16" bibid="bib27" firstref="ref27"></nolink> <nolink nlid="nl17" bibid="bib28" firstref="ref29"></nolink> <nolink nlid="nl18" bibid="bib30" firstref="ref30"></nolink> <nolink nlid="nl19" bibid="bib32" firstref="ref31"></nolink> <nolink nlid="nl20" bibid="bib31" firstref="ref32"></nolink> <nolink nlid="nl21" bibid="bib34" firstref="ref38"></nolink> <nolink nlid="nl22" bibid="bib35" firstref="ref39"></nolink> <nolink nlid="nl23" bibid="bib36" firstref="ref41"></nolink> <nolink nlid="nl24" bibid="bib37" firstref="ref42"></nolink> <nolink nlid="nl25" bibid="bib38" firstref="ref43"></nolink> <nolink nlid="nl26" bibid="bib39" firstref="ref44"></nolink> <nolink nlid="nl27" bibid="bib40" firstref="ref47"></nolink> <nolink nlid="nl28" bibid="bib41" firstref="ref48"></nolink> <nolink nlid="nl29" bibid="bib42" firstref="ref51"></nolink> <nolink nlid="nl30" bibid="bib43" firstref="ref52"></nolink> <nolink nlid="nl31" bibid="bib44" firstref="ref53"></nolink> <nolink nlid="nl32" bibid="bib45" firstref="ref60"></nolink> <nolink nlid="nl33" bibid="bib46" firstref="ref61"></nolink> <nolink nlid="nl34" bibid="bib47" firstref="ref70"></nolink> <nolink nlid="nl35" bibid="bib48" firstref="ref77"></nolink> <nolink nlid="nl36" bibid="bib49" firstref="ref85"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1253063
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Random-Effects Meta-Analysis of Few Studies Involving Rare Events
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Günhan%2C+Burak+Kürsad%22">Günhan, Burak Kürsad</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-7454-8680">0000-0002-7454-8680</externalLink>)<br /><searchLink fieldCode="AR" term="%22Röver%2C+Christian%22">Röver, Christian</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-6911-698X">0000-0002-6911-698X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Friede%2C+Tim%22">Friede, Tim</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5347-7441">0000-0001-5347-7441</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Research+Synthesis+Methods%22"><i>Research Synthesis Methods</i></searchLink>. Jan 2020 11(1):74-90.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 17
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2020
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Information Analyses
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Meta+Analysis%22">Meta Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+Research%22">Medical Research</searchLink><br /><searchLink fieldCode="DE" term="%22Drug+Therapy%22">Drug Therapy</searchLink><br /><searchLink fieldCode="DE" term="%22Bayesian+Statistics%22">Bayesian Statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Probability%22">Probability</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Analysis%22">Data Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Databases%22">Databases</searchLink><br /><searchLink fieldCode="DE" term="%22Simulation%22">Simulation</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Bias%22">Statistical Bias</searchLink><br /><searchLink fieldCode="DE" term="%22Intervals%22">Intervals</searchLink><br /><searchLink fieldCode="DE" term="%22Safety%22">Safety</searchLink><br /><searchLink fieldCode="DE" term="%22Pediatrics%22">Pediatrics</searchLink><br /><searchLink fieldCode="DE" term="%22Surgery%22">Surgery</searchLink><br /><searchLink fieldCode="DE" term="%22Physiology%22">Physiology</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Software%22">Computer Software</searchLink><br /><searchLink fieldCode="DE" term="%22Models%22">Models</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1002/jrsm.1370
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1759-2879
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Meta-analyses of clinical trials targeting rare events face particular challenges when the data lack adequate numbers of events for all treatment arms. Especially when the number of studies is low, standard random-effects meta-analysis methods can lead to serious distortions because of such data sparsity. To overcome this, we suggest the use of "weakly informative priors" (WIPs) for the treatment effect parameter of a Bayesian meta-analysis model, which may also be seen as a form of penalization. As a data model, we use a binomial-normal hierarchical model (BNHM) that does not require continuity corrections in case of zero counts in one or both arms. We suggest a normal prior for the log-odds ratio with mean 0 and standard deviation 2.82, which is motivated (a) as a symmetric prior centered around unity and constraining the odds ratio within a range from 1/250 to 250 with 95% probability and (b) as consistent with empirically observed effect estimates from a set of 37 773 meta-analyses from the Cochrane Database of Systematic Reviews. In a simulation study with rare events and few studies, our BNHM with a WIP outperformed a Bayesian method without a WIP and a maximum likelihood estimator in terms of smaller bias and shorter interval estimates with similar coverage. Furthermore, the methods are illustrated by a systematic review in immunosuppression of rare safety events following pediatric transplantation. A publicly available R package, MetaStan, is developed to automate a Bayesian implementation of meta-analysis models using WIPs.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2020
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1253063
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1253063
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1002/jrsm.1370
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 17
        StartPage: 74
    Subjects:
      – SubjectFull: Meta Analysis
        Type: general
      – SubjectFull: Medical Research
        Type: general
      – SubjectFull: Drug Therapy
        Type: general
      – SubjectFull: Bayesian Statistics
        Type: general
      – SubjectFull: Probability
        Type: general
      – SubjectFull: Data Analysis
        Type: general
      – SubjectFull: Databases
        Type: general
      – SubjectFull: Simulation
        Type: general
      – SubjectFull: Statistical Bias
        Type: general
      – SubjectFull: Intervals
        Type: general
      – SubjectFull: Safety
        Type: general
      – SubjectFull: Pediatrics
        Type: general
      – SubjectFull: Surgery
        Type: general
      – SubjectFull: Physiology
        Type: general
      – SubjectFull: Computer Software
        Type: general
      – SubjectFull: Models
        Type: general
    Titles:
      – TitleFull: Random-Effects Meta-Analysis of Few Studies Involving Rare Events
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Günhan, Burak Kürsad
      – PersonEntity:
          Name:
            NameFull: Röver, Christian
      – PersonEntity:
          Name:
            NameFull: Friede, Tim
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2020
          Identifiers:
            – Type: issn-print
              Value: 1759-2879
          Numbering:
            – Type: volume
              Value: 11
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Research Synthesis Methods
              Type: main
ResultId 1