View in EDS HTML Full Text PDF Full Text

Accounting for Time Dependency in Meta-Analyses of Concordance Probability Estimates

Saved in:
Bibliographic Details
Title: Accounting for Time Dependency in Meta-Analyses of Concordance Probability Estimates
Language: English
Authors: Schmid, Matthias (ORCID 0000-0002-0788-0317), Friede, Tim (ORCID 0000-0001-5347-7441), Klein, Nadja (ORCID 0000-0002-5072-5347), Weinhold, Leonie
Source: Research Synthesis Methods. 2023 14(6):807-823.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 17
Publication Date: 2023
Document Type: Journal Articles
Reports - Research
Descriptors: Probability, Meta Analysis, Time, Test Validity, Bias
DOI: 10.1002/jrsm.1655
ISSN: 1759-2879
1759-2887
Abstract: Recent years have seen the development of many novel scoring tools for disease prognosis and prediction. To become accepted for use in clinical applications, these tools have to be validated on external data. In practice, validation is often hampered by logistical issues, resulting in multiple small-sized validation studies. It is therefore necessary to synthesize the results of these studies using techniques for meta-analysis. Here we consider strategies for meta analyzing the concordance probability for time-to-event data ("C-index"), which has become a popular tool to evaluate the discriminatory power of prediction models with a right-censored outcome. We show that standard meta-analysis of the C-index may lead to biased results, as the magnitude of the concordance probability depends on the length of the time interval used for evaluation (defined e.g., by the follow-up time, which might differ considerably between studies). To address this issue, we propose a set of methods for random-effects meta-regression that incorporate time directly as covariate in the model equation. In addition to analyzing nonlinear time trends via fractional polynomial, spline, and exponential decay models, we provide recommendations on suitable transformations of the C-index before meta-regression. Our results suggest that the C-index is best meta-analyzed using fractional polynomial meta-regression with logit-transformed C-index values. Classical random-effects meta-analysis (not considering time as covariate) is demonstrated to be a suitable alternative when follow-up times are small. Our findings have implications for the reporting of C-index values in future studies, which should include information on the length of the time interval underlying the calculations.
Abstractor: As Provided
Notes: https://github.com/weinholdl/metac
Entry Date: 2023
Accession Number: EJ1399283
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHdU3ku38LGUIUHysGVS5-dAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDIqUCzJ6Cihu0wHV5AIBEICBm_5b7hmyjqlObfQX1R8UX9GsWg6MCOkkaPjEeAJd_mcfbUXn-_xvUkCrimOfRe0t0pZLujqK4GdcmomdvuQw5a0Oll7QLgE2nWBjd5O3IM0gfTMG6Nx8JO0rYT-V3xUZCK7uqU_daLVKfHAg4y-ZkbVW83eiuNyA6yvZD6hyRA6MgvE3ALRa8187gwNCgxlK8upM_pACfRjRnvxl
Text:
  Availability: 1
  Value: <anid>AN0173455155;[bdct]01nov.23;2023Nov08.05:16;v2.2.500</anid> <title id="AN0173455155-1">Accounting for time dependency in meta‐analyses of concordance probability estimates </title> <p>Recent years have seen the development of many novel scoring tools for disease prognosis and prediction. To become accepted for use in clinical applications, these tools have to be validated on external data. In practice, validation is often hampered by logistical issues, resulting in multiple small‐sized validation studies. It is therefore necessary to synthesize the results of these studies using techniques for meta‐analysis. Here we consider strategies for meta analyzing the concordance probability for time‐to‐event data ("C‐index"), which has become a popular tool to evaluate the discriminatory power of prediction models with a right‐censored outcome. We show that standard meta‐analysis of the C‐index may lead to biased results, as the magnitude of the concordance probability depends on the length of the time interval used for evaluation (defined e.g., by the follow‐up time, which might differ considerably between studies). To address this issue, we propose a set of methods for random‐effects meta‐regression that incorporate time directly as covariate in the model equation. In addition to analyzing nonlinear time trends via fractional polynomial, spline, and exponential decay models, we provide recommendations on suitable transformations of the C‐index before meta‐regression. Our results suggest that the C‐index is best meta‐analyzed using fractional polynomial meta‐regression with logit‐transformed C‐index values. Classical random‐effects meta‐analysis (not considering time as covariate) is demonstrated to be a suitable alternative when follow‐up times are small. Our findings have implications for the reporting of C‐index values in future studies, which should include information on the length of the time interval underlying the calculations.</p> <p>Keywords: concordance probability; fractional polynomials; meta‐regression; prognostic factor research; restricted cubic splines; time‐to‐event data</p> <hd id="AN0173455155-2">Highlights</hd> <p></p> <hd id="AN0173455155-3">What is already known?</hd> <p>Prognostic factor research is a rapidly evolving field with an increased need for meta‐analysis. In this field, studies aim at analyzing the associations of risk factors with a time‐to‐event outcome. The concordance probability for time‐to‐event data ("<emph>C</emph>‐index") has become a popular tool to evaluate the discriminatory power of prognostic models. It is often used in the meta‐analysis of validation studies.</p> <hd id="AN0173455155-4">What is new?</hd> <p>We show that standard meta‐analysis of the <emph>C</emph>‐index may lead to biased results, as the magnitude of the <emph>C</emph>‐index depends on the length of the time interval used for evaluation. To address this issue, we propose a set of methods for random‐effects meta‐regression incorporating time directly as covariate in the model equation. Our methods are able to account for nonlinear relationships between the <emph>C</emph>‐index and time.</p> <hd id="AN0173455155-5">Potential impact for readers outside the authors' field</hd> <p>The proposed methods improve the interpretability of meta‐analyzed prognostic models, enabling users of these models to better judge their prognostic power. They also point at a possible overoptimism in the interpretation of prognostic models, which—if evaluated by the <emph>C</emph>‐index—could indicate a strong performance simply because the time interval for evaluation has been (too) short. Furthermore, our results have implications for the reporting of future validation studies. These should present the full <emph>C</emph>‐index curve, along with the time interval used for evaluation.</p> <hd id="AN0173455155-6">INTRODUCTION</hd> <p>During the past decades, the volume of published research has increased dramatically.[<reflink idref="bib1" id="ref1">1</reflink>] Even before the COVID‐19 pandemic, the number of research articles has been estimated to grow by 8%–9% each year, including more than 1 million papers per year in the biomedical field alone.[<reflink idref="bib2" id="ref2">2</reflink>] At the same time, hundreds of newly ranked journals have appeared, with the estimated total amount of active peer‐reviewed journals exceeding 30,000.[[<reflink idref="bib1" id="ref3">1</reflink>], [<reflink idref="bib3" id="ref4">3</reflink>]] In view of this "information overload",[<reflink idref="bib2" id="ref5">2</reflink>] there is an obvious need for evidence synthesis to "clarify what is known from research evidence to inform policy, practice and personal decision making and improved methods for meta‐analysis".[<reflink idref="bib4" id="ref6">4</reflink>]</p> <p> <emph>Prognostic factor research</emph>[<reflink idref="bib5" id="ref7">5</reflink>] is a rapidly evolving field with an increased need for meta‐analysis. In this field, studies aim at analyzing the associations of one or several factors (often termed "risk factors") with a time‐to‐event outcome <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0001" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>T</mi><mo>∈</mo><msup><mi mathvariant="normal">ℝ</mi><mo>+</mo></msup></mrow></math> </ephtml> . In medicine and epidemiology, for instance, prognostic factors are often given by patient characteristics (e.g. age, sex, smoking behavior, blood pressure) collected at the baseline examination of a longitudinal study. These variables might then be used to predict the occurrence of events such as death, tumor progression, or adverse events. Often, several prognostic factors are summarized by a multivariable risk score (defined, e.g., by a linear combination of the factors). Popular examples of risk scores are the European System for Cardiac Operative Risk Evaluation (EuroSCORE) II to predict mortality after cardiac surgery and the Framingham Risk Score for predicting coronary heart disease.[<reflink idref="bib6" id="ref8">6</reflink>] Score development is usually based on a statistical modeling technique applied to a set of training data, yielding a prediction model that is defined by a (univariable or multivariable) prognostic score <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0002" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>η</mi><mo>∈</mo><mi mathvariant="normal">ℝ</mi></mrow></math> </ephtml> .</p> <p>A key issue for the acceptability of a prognostic score is its repeated validation on externally collected test data.[[<reflink idref="bib7" id="ref9">7</reflink>]] These validation steps have become a gold standard in prognostic modeling, as they provide a much more realistic assessment of the score's performance than would have been possible using the training data only. Importantly, the results of external validation steps are often found to be heterogeneous, showing a high variability in prognostic performance. Validation studies involving external test data might, for instance, be affected by small sample sizes and differences in the characteristics of the patient population compared to the training data.[[<reflink idref="bib6" id="ref10">6</reflink>], [<reflink idref="bib9" id="ref11">9</reflink>]] As a consequence, systematic reviews and meta‐analyses are "urgently needed to summarize [the] evidence [of prediction models] and to better understand under what circumstances developed models perform adequately or require further adjustments".[<reflink idref="bib6" id="ref12">6</reflink>]</p> <p>In this paper, we consider strategies for meta analyzing the <emph>concordance probability for time‐to‐event data</emph> ("<emph>C</emph>‐index"), which is a widely used measure to evaluate prediction models with a time‐to‐event outcome.[[<reflink idref="bib10" id="ref13">10</reflink>], [<reflink idref="bib12" id="ref14">12</reflink>]] The <emph>C</emph>‐index is a discrimination measure that compares the rankings of the individual score values <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0003" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>η</mi><mi>i</mi></msub></mrow></math> </ephtml> and the event times <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0004" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>T</mi><mi>i</mi></msub></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0005" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mi>n</mi></mrow></math> </ephtml> , in a test sample of size <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0006" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>n</mi></mrow></math> </ephtml> . It is defined[<reflink idref="bib12" id="ref15">12</reflink>] by1 <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0007" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced><mo linebreak="goodbreak">=</mo><mi mathvariant="normal">P</mi><mspace width="0.1em" /><mfenced open="(" close=")" separators="|,"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>></mo><msub><mi>η</mi><mi>j</mi></msub><mspace width="0.1em" /></mrow><mrow><mspace width="0.1em" /><msub><mi>T</mi><mi>i</mi></msub><mo><</mo><msub><mi>T</mi><mi>j</mi></msub></mrow><mrow><msub><mi>T</mi><mi>i</mi></msub><mo>≤</mo><mi>τ</mi></mrow></mfenced><mo>,</mo></mrow></math> </ephtml> where <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0008" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>i</mi><mo>,</mo><mi>j</mi></mrow></math> </ephtml> denote two independent observations in the test data and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0009" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi><mo>></mo><mn>0</mn></mrow></math> </ephtml> is a truncation time (e.g., the maximum follow‐up time of a clinical study). Setting <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0010" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi><mo>=</mo><mi>∞</mi></mrow></math> </ephtml> yields the <emph>unrestricted concordance probability</emph><ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0011" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><mi mathvariant="normal">P</mi><mspace width="0.1em" /><mfenced open="(" close=")" separators="|"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>></mo><msub><mi>η</mi><mi>j</mi></msub><mspace width="0.1em" /></mrow><mrow><mspace width="0.1em" /><msub><mi>T</mi><mi>i</mi></msub><mo><</mo><msub><mi>T</mi><mi>j</mi></msub></mrow></mfenced></mrow></math> </ephtml> . Generally, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0012" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>∞</mi></mfenced></mrow></math> </ephtml> takes the value 1 if the rankings of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0013" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>−</mo><msub><mi>η</mi><mi>i</mi></msub></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0014" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>T</mi><mi>i</mi></msub></mrow></math> </ephtml> agree perfectly. Conversely, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0015" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>∞</mi></mfenced><mo>=</mo><mn>0.5</mn></mrow></math> </ephtml> if <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0016" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>η</mi></mrow></math> </ephtml> does not predict better than chance alone. In the absence of censoring, the concordance probability can readily be evaluated by comparing all pairs <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0017" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>T</mi><mi>i</mi></msub><mo>,</mo><msub><mi>T</mi><mi>j</mi></msub></mrow></math> </ephtml> and by estimating the conditional probability in (<reflink idref="bib1" id="ref16">1</reflink>) by its respective relative frequency in the test data. If censoring is present, however, a comparison of all pairs <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0018" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>T</mi><mi>i</mi></msub><mo>,</mo><msub><mi>T</mi><mi>j</mi></msub></mrow></math> </ephtml> is no longer possible, and estimation of the concordance probability requires additional assumptions on the data‐generating process,[[<reflink idref="bib11" id="ref17">11</reflink>], [<reflink idref="bib13" id="ref18">13</reflink>]] also see Schmid and Potapov[<reflink idref="bib15" id="ref19">15</reflink>] for a comparison of estimators.</p> <p>For meta‐analysis, Debray et al.[<reflink idref="bib6" id="ref20">6</reflink>] recently introduced a framework that includes, among other techniques, a method to summarize estimates of the <emph>C</emph>‐index obtained from multiple validation studies. Based on earlier work by Snell et al., who meta‐analyzed the QRISK2 model to predict 10‐year cardiovascular disease risk,[<reflink idref="bib16" id="ref21">16</reflink>] the authors proposed to transform estimates to the logit scale before meta‐analysis. This strategy has also been adopted in several recent systematic reviews and meta‐analyses,[[<reflink idref="bib17" id="ref22">17</reflink>], [<reflink idref="bib19" id="ref23">19</reflink>]] including evaluations of the CHA2DS2‐VASc rule for estimating stroke risk in patients with atrial fibrillation and various scores for the prediction of survival outcomes in colorectal cancer with surgical resection. In other studies, which included meta‐analyses of models for survival after resection of intrahepatic cholangiocarcinoma, the <emph>C</emph>‐index was meta‐analyzed on the original (untransformed) probability scale.[[<reflink idref="bib20" id="ref24">20</reflink>]] Meta‐analysis of the <emph>C</emph>‐index using individual participant data has been studied by Pennells et al.[<reflink idref="bib22" id="ref25">22</reflink>] Hattori and Zhou[<reflink idref="bib23" id="ref26">23</reflink>] proposed to construct a synthesized <emph>C</emph>‐index from an estimate of the summary cumulative ROC curve obtained by analyzing study‐specific Kaplan–Meier curves.</p> <p>Despite numerous methodological advances, which have led to the publication of several guidance papers,[[<reflink idref="bib6" id="ref27">6</reflink>], [<reflink idref="bib9" id="ref28">9</reflink>]] meta‐analysis of prognostic validation studies remains a challenging task. This is, in particular, due to the fact that measures of prediction accuracy in prognostic research are often related to a specific time point or time span.[<reflink idref="bib5" id="ref29">5</reflink>] Consequently, meta‐analysis of validation studies becomes intrinsically difficult when study‐specific performance estimates refer to different time points or spans. As seen from (<reflink idref="bib1" id="ref30">1</reflink>), this time dependency also affects the <emph>C</emph>‐index studied in this paper: Since the magnitude of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0019" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi></mrow></math> </ephtml> depends on the truncation time <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0020" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi></mrow></math> </ephtml> , <emph>C</emph>‐index estimates may not be comparable across studies if they relate to different values of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0021" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi></mrow></math> </ephtml> . Specifically, since the value of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0022" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi></mrow></math> </ephtml> is often determined by the duration of the study generating the test data, different study durations may implicitly lead to systematic differences between the resulting <emph>C</emph>‐index estimates. Consider, for instance, the simulated meta‐analysis in Figure 1: In this example, we considered 30 hypothetical studies, generating event times from a Weibull accelerated failure time model of the form <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0023" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>log</mi><mfenced open="(" close=")"><mi>T</mi></mfenced><mo>=</mo><mi>X</mi><mo>−</mo><mi mathvariant="normal">ϵ</mi></mrow></math> </ephtml> . In each study, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0024" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>X</mi></mrow></math> </ephtml> was a normally distributed covariate with zero mean and standard deviation 0.5, and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0025" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi mathvariant="normal">ϵ</mi></mrow></math> </ephtml> followed a standard Gumbel distribution. Censoring times were independent of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0026" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>T</mi></mrow></math> </ephtml> and followed an exponential distribution with rate 0.5 each. Sample sizes of the studies were generated randomly and ranged between 100 and 1000. After data generation, the observed event times were truncated at study‐specific truncation (=maximum follow‐up) times <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0027" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0028" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>k</mi><mo>=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mn>30</mn></mrow></math> </ephtml> , which were sampled from a uniform distribution on <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0029" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mfenced open="[" close="]"><mn>0.1,2</mn></mfenced></mrow></math> </ephtml> . Study‐specific <emph>C</emph>‐index estimates were calculated using the estimator by Uno et al.[<reflink idref="bib14" id="ref31">14</reflink>] As demonstrated in Figure 1, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0030" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> is seen to decrease with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0031" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi></mrow></math> </ephtml> , and the pooled estimate obtained from standard meta‐analysis relates to an implicitly defined truncation time. Thus, if not accounted for, the time dependency of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0032" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> may compromise both the specification of a properly defined estimand <emph>and</emph> the validity of the pooled estimate. While the simulated meta‐analysis in Figure 1 serves as an illustrative example to motivate the methods proposed here, very similar effects have also been found in real‐world studies (see e.g., our analysis of the German Chronic Kidney Disease Study[<reflink idref="bib24" id="ref32">24</reflink>] data in Section 4 of this paper). Another recent example is given by the <emph>C</emph>‐index estimates presented in Zacharias et al.,[<reflink idref="bib25" id="ref33">25</reflink>] who evaluated two prediction models for the progression of chronic kidney disease to kidney failure. Although the authors considered a slightly different setting than the one introduced above (allowing for the presence of a competing event), the resulting <emph>C</emph>‐index estimates (computed in three external validation studies and presented in table 2 of Zacharias et al.) show a clear negative association with the truncation time.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01nov23/jrsm1655-fig-0001.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1655-fig-0001.jpg" title="1 Exemplary meta‐analysis with simulated test data. The upper panel shows the expected censoring rate at each value of the truncation time. The lower panel shows the study‐specific C‐index estimates. The sizes of the bubbles are proportional to the inverse variances of the C‐index estimates. The solid black line refers to the true C‐index according to the data‐generating process whereas the horizontal gray line refers to the pooled C‐index estimate that would have been obtained from a standard random effects meta‐analysis ignoring time dependency. The vertical dashed line shows the "implicit" truncation time corresponding to the pooled estimate. Obviously, this model lacks a well‐defined estimand, and it is unclear how the pooled estimate should be interpreted. The blue and red lines refer to the meta‐regression curves obtained from fitting a restricted cubic spline and a fractional polynomial model to the logit‐transformed C‐index estimates. For details on model specification, see Section 3. [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <p>To address these issues and to improve the interpretability of pooled <emph>C</emph>‐index estimates, we consider a set of statistical techniques that incorporate the time dependency of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0033" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> directly in a suitably specified meta‐regression model. Our proposed model is based on the frequentist modeling approach with restricted maximum likelihood (REML) estimation, as recommended in the recent guidance paper by Debray et al.[<reflink idref="bib6" id="ref34">6</reflink>] Specifically, due to the above‐mentioned heterogeneity of external validation results, we will focus throughout on random‐effects models. We propose to model the time dependency of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0034" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> by either a restricted cubic spline (RCS) or a 2nd degree fractional polynomial (FP2), thereby accounting for nonlinearities in the regression curve (cf. Figure 1). Using simulation studies, we will compare the RCS and FP2 models to standard random‐effects meta‐analysis not including <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0035" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi></mrow></math> </ephtml> as covariate, and also to linear meta‐regression. Furthermore, we will investigate whether meta‐regression can be improved by transforming the <emph>C</emph>‐index estimates before model fitting (for instance, using a logit transformation).</p> <p>The rest of the paper is organized as follows: After starting with the definition of relevant quantities (Section 2.1), we provide a brief overview of existing techniques to estimate the <emph>C</emph>‐index (Section 2.2). The proposed methodology is described in Sections 2.3 and 2.4. Section 3 contains a comprehensive simulation study on the properties of the proposed approach, including a comparison to existing methods. A real‐world illustration on data collected for the German Chronic Kidney Disease Study is presented in Section 4. The final section summarizes the main findings of the article.</p> <hd id="AN0173455155-8">METHODS</hd> <p></p> <hd id="AN0173455155-9">Derivation and properties of the C ‐index</hd> <p>Consider a validation study with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0036" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>n</mi></mrow></math> </ephtml> observations and a time‐to‐event outcome that might be subject to right censoring. The observations are assumed to be independent and identically distributed. The score values and observed event times are denoted by <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0037" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>η</mi><mi>i</mi></msub></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0038" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo>=</mo><mi>min</mi><mfenced open="(" close=")" separators=","><msub><mi>T</mi><mi>i</mi></msub><msub><mi>Z</mi><mi>i</mi></msub></mfenced></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0039" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mi>n</mi></mrow></math> </ephtml> , respectively, where <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0040" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msup><mfenced open="(" close=")" separators=",,"><msub><mi>Z</mi><mn>1</mn></msub><mi>...</mi><msub><mi>Z</mi><mi>n</mi></msub></mfenced><mi>Τ</mi></msup></mrow></math> </ephtml> is a vector of continuous censoring times. The binary variables <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0041" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi mathvariant="normal">Δ</mi><mi>i</mi></msub><mo>=</mo><mi>I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mi>T</mi><mi>i</mi></msub><mo>≤</mo><msub><mi>Z</mi><mi>i</mi></msub></mrow></mfenced></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0042" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mi>n</mi></mrow></math> </ephtml> , indicate whether observations are censored ( <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0043" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi mathvariant="normal">Δ</mi><mi>i</mi></msub><mo>=</mo><mn>0</mn></mrow></math> </ephtml> ) or not ( <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0044" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi mathvariant="normal">Δ</mi><mi>i</mi></msub><mo>=</mo><mn>1</mn></mrow></math> </ephtml> ). Assumptions on the censoring process are given below. We further assume that there are no tied observations, i.e. all sample values of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0045" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>T</mi><mi>i</mi></msub></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0046" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>Z</mi><mi>i</mi></msub></mrow></math> </ephtml> are assumed to be unique. As shown by Heagerty and Zheng,[<reflink idref="bib11" id="ref35">11</reflink>] the concordance probability in (<reflink idref="bib1" id="ref36">1</reflink>) can be derived from a set of time‐dependent sensitivities and specificities, which, at each time point <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0047" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>t</mi></mrow></math> </ephtml> , relate the current survival status to the event that <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0048" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>η</mi></mrow></math> </ephtml> exceeds a given threshold <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0049" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>c</mi><mo>∈</mo><mi mathvariant="normal">ℝ</mi></mrow></math> </ephtml> . More specifically, following the <emph>incident/dynamic</emph> approach,[<reflink idref="bib11" id="ref37">11</reflink>] one defines incident cases by observations experiencing an event at <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0050" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>t</mi></mrow></math> </ephtml> (i.e., <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0051" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>T</mi><mi>i</mi></msub><mo>=</mo><mi>t</mi></mrow></math> </ephtml> ) and dynamic controls by observations having the event after <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0052" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>t</mi></mrow></math> </ephtml> (i.e., <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0053" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>T</mi><mi>i</mi></msub><mo>></mo><mi>t</mi></mrow></math> </ephtml> ). With these definitions, time‐dependent sensitivities and specificities are given by <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0054" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><msubsup><mtext>sens</mtext><mi>t</mi><mi>I</mi></msubsup><mfenced open="(" close=")"><mi>c</mi></mfenced><mo linebreak="goodbreak">=</mo><mi mathvariant="normal">P</mi><mspace width="0.1em" /><mfenced open="(" close=")" separators="|"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>></mo><mi>c</mi><mspace width="0.1em" /></mrow><mrow><mspace width="0.1em" /><msub><mi>T</mi><mi>i</mi></msub><mo linebreak="goodbreak">=</mo><mi>t</mi></mrow></mfenced><mspace width="1.1em" /><mtext>and</mtext></mrow></math> </ephtml><ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0055" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><msubsup><mtext>spec</mtext><mi>t</mi><mi>D</mi></msubsup><mfenced open="(" close=")"><mi>c</mi></mfenced><mo linebreak="goodbreak">=</mo><mi mathvariant="normal">P</mi><mspace width="0.1em" /><mfenced open="(" close=")" separators="|"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>≤</mo><mi>c</mi><mspace width="0.1em" /></mrow><mrow><mspace width="0.1em" /><msub><mi>T</mi><mi>i</mi></msub><mo>></mo><mi>t</mi></mrow></mfenced><mo>,</mo></mrow></math> </ephtml> where the superscripts <emph>I</emph> and <emph>D</emph> refer to the terms "incident" and "dynamic", respectively. At each time point, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0056" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><msubsup><mtext>sens</mtext><mi>t</mi><mi>I</mi></msubsup><mfenced open="(" close=")"><mi>c</mi></mfenced></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0057" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><msubsup><mtext>spec</mtext><mi>t</mi><mi>D</mi></msubsup><mfenced open="(" close=")"><mi>c</mi></mfenced></mrow></math> </ephtml> can be summarized by an incident/dynamic receiver operating characteristic (ROC) curve, which is defined as2 <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0058" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><msubsup><mi>ROC</mi><mi>t</mi><mrow><mi>I</mi><mo>/</mo><mi>D</mi></mrow></msubsup><mfenced open="(" close=")"><mi>p</mi></mfenced><mo linebreak="goodbreak">=</mo><msubsup><mtext>sens</mtext><mi>t</mi><mi>I</mi></msubsup><mfenced open="[" close="]"><mrow><msup><mfenced open="(" close=")"><mrow><mn>1</mn><mo linebreak="goodbreak">−</mo><msubsup><mtext>spec</mtext><mi>t</mi><mi>D</mi></msubsup></mrow></mfenced><mrow><mo>−</mo><mn>1</mn></mrow></msup><mfenced open="(" close=")"><mi>p</mi></mfenced></mrow></mfenced><mo>,</mo><mspace width="1em" /><mi>p</mi><mo>∈</mo><mfenced open="[" close="]"><mrow><mn>0</mn><mo>,</mo><mn>1</mn></mrow></mfenced><mo>.</mo></mrow></math> </ephtml></p> <p>Incident/dynamic ROC curves can further be summarized by the incident/dynamic AUC curve <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0059" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><msubsup><mi>AUC</mi><mi>t</mi><mrow><mi>I</mi><mo>/</mo><mi>D</mi></mrow></msubsup><mo linebreak="goodbreak">=</mo><msubsup><mo>∫</mo><mn>0</mn><mn>1</mn></msubsup><mspace width="0.1em" /><msubsup><mi>ROC</mi><mi>t</mi><mrow><mi>I</mi><mo>/</mo><mi>D</mi></mrow></msubsup><mfenced open="(" close=")"><mi>p</mi></mfenced><mspace width="0.1em" /><mi mathvariant="italic">dp</mi><mo>,</mo></mrow></math> </ephtml> which equals the probability <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0060" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><mi mathvariant="normal">P</mi><mspace width="0.1em" /><mfenced open="(" close=")" separators="|,"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>></mo><msub><mi>η</mi><mi>j</mi></msub><mspace width="0.1em" /></mrow><mrow><mspace width="0.1em" /><msub><mi>T</mi><mi>i</mi></msub><mo>=</mo><mi>t</mi></mrow><mrow><msub><mi>T</mi><mi>j</mi></msub><mo>></mo><mi>t</mi></mrow></mfenced></mrow></math> </ephtml> for independent observations <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0061" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>i</mi></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0062" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>j</mi></mrow></math> </ephtml> . Finally, denoting the probability density function of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0063" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>T</mi></mrow></math> </ephtml> by <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0064" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>f</mi><mfenced open="(" close=")"><mi>t</mi></mfenced></mrow></math> </ephtml> , the concordance probability <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0065" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> is derived as the area under a weighted version of the incident/dynamic AUC curve. More specifically, it can be shown that3 <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0066" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced><mo linebreak="goodbreak">=</mo><mi mathvariant="normal">P</mi><mspace width="0.1em" /><mfenced open="(" close=")" separators="|,"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>></mo><msub><mi>η</mi><mi>j</mi></msub><mspace width="0.1em" /></mrow><mrow><mspace width="0.1em" /><msub><mi>T</mi><mi>i</mi></msub><mo><</mo><msub><mi>T</mi><mi>j</mi></msub></mrow><mrow><msub><mi>T</mi><mi>i</mi></msub><mo>≤</mo><mi>τ</mi></mrow></mfenced><mo linebreak="goodbreak">=</mo><msubsup><mo>∫</mo><mn>0</mn><mi>τ</mi></msubsup><msubsup><mi>w</mi><mi>t</mi><mi>τ</mi></msubsup><mo>⋅</mo><msubsup><mi>AUC</mi><mi>t</mi><mrow><mi>I</mi><mo>/</mo><mi>D</mi></mrow></msubsup><mspace width="0.1em" /><mi mathvariant="italic">dt</mi></mrow></math> </ephtml> with weights <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0067" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>w</mi><mi>t</mi><mi>τ</mi></msubsup><mo>=</mo><mi>f</mi><mfenced open="(" close=")"><mi>t</mi></mfenced><mo>⋅</mo><mi mathvariant="normal">P</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><mi>T</mi><mo>></mo><mi>t</mi></mrow></mfenced><mo>/</mo><msubsup><mo>∫</mo><mn>0</mn><mi>τ</mi></msubsup><mi>f</mi><mfenced open="(" close=")"><mi>u</mi></mfenced><mo>⋅</mo><mi>P</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><mi>T</mi><mo>></mo><mi>u</mi></mrow></mfenced><mspace width="0.1em" /><mi mathvariant="italic">du</mi></mrow></math> </ephtml> (see Heagerty and Zheng[<reflink idref="bib11" id="ref38">11</reflink>] for a formal proof).</p> <p>A related quantity is the <emph>cumulative/dynamic</emph> ROC curve, which is defined in the same way as (<reflink idref="bib2" id="ref39">2</reflink>) but with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0068" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><msubsup><mtext>sens</mtext><mi>t</mi><mi>I</mi></msubsup><mfenced open="(" close=")"><mi>c</mi></mfenced></mrow></math> </ephtml> replaced by time‐dependent sensitivities of the form <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0069" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><msubsup><mtext>sens</mtext><mi>t</mi><mi>C</mi></msubsup><mfenced open="(" close=")"><mi>c</mi></mfenced><mo>=</mo><mi mathvariant="normal">P</mi><mspace width="0.1em" /><mfenced open="(" close=")" separators="|"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>></mo><mi>c</mi><mspace width="0.1em" /></mrow><mrow><mspace width="0.1em" /><msub><mi>T</mi><mi>i</mi></msub><mo>≤</mo><mi>t</mi></mrow></mfenced></mrow></math> </ephtml> , where the superscript <emph>C</emph> refers to the term "cumulative". With this approach, <emph>cumulative cases</emph> are defined by observations experiencing an event at or before <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0070" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>t</mi></mrow></math> </ephtml> (i.e., <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0071" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>T</mi><mi>i</mi></msub><mo>≤</mo><mi>t</mi></mrow></math> </ephtml> ). Correspondingly, the cumulative/dynamic AUC curve is given by the areas under the cumulative/dynamic ROC curves, i.e. by <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0072" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><msubsup><mi>AUC</mi><mi>t</mi><mrow><mi>C</mi><mo>/</mo><mi>D</mi></mrow></msubsup><mo>=</mo><mi mathvariant="normal">P</mi><mspace width="0.1em" /><mfenced open="(" close=")" separators="|,"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>></mo><msub><mi>η</mi><mi>j</mi></msub><mspace width="0.1em" /></mrow><mrow><mspace width="0.1em" /><msub><mi>T</mi><mi>i</mi></msub><mo>≤</mo><mi>t</mi></mrow><mrow><msub><mi>T</mi><mi>j</mi></msub><mo>></mo><mi>t</mi></mrow></mfenced></mrow></math> </ephtml> . Defining a generalized version of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0073" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><msubsup><mi>AUC</mi><mi>t</mi><mrow><mi>C</mi><mo>/</mo><mi>D</mi></mrow></msubsup></mrow></math> </ephtml> by <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0074" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><msubsup><mi>AUC</mi><mrow><mi>s</mi><mo>,</mo><mi>t</mi></mrow><mrow><mi>C</mi><mo>/</mo><mi>D</mi></mrow></msubsup><mo>=</mo><mi mathvariant="normal">P</mi><mspace width="0.1em" /><mfenced open="(" close=")" separators="|,"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>></mo><msub><mi>η</mi><mi>j</mi></msub><mspace width="0.1em" /></mrow><mrow><mspace width="0.1em" /><msub><mi>T</mi><mi>i</mi></msub><mo>≤</mo><mi>s</mi></mrow><mrow><msub><mi>T</mi><mi>j</mi></msub><mo>></mo><mi>t</mi></mrow></mfenced></mrow></math> </ephtml> , it can further be shown[<reflink idref="bib23" id="ref40">23</reflink>] that4 <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0075" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><msubsup><mi>AUC</mi><mi>t</mi><mrow><mi>I</mi><mo>/</mo><mi>D</mi></mrow></msubsup><mo linebreak="goodbreak">=</mo><mfrac><mi>∂</mi><mrow><mi>∂</mi><mi>s</mi></mrow></mfrac><msub><mfenced open="" close="|"><mrow><mspace width="0.1em" /><msubsup><mi>AUC</mi><mrow><mi>s</mi><mo>,</mo><mi>t</mi></mrow><mrow><mi>C</mi><mo>/</mo><mi>D</mi></mrow></msubsup><mspace width="0.1em" /></mrow></mfenced><mrow><mi>s</mi><mo>=</mo><mi>t</mi></mrow></msub><mo>⋅</mo><mfrac><mrow><mspace width="0.1em" /><mi mathvariant="normal">P</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><mi>T</mi><mo>≤</mo><mi>t</mi></mrow></mfenced></mrow><mrow><mi>f</mi><mfenced open="(" close=")"><mi>t</mi></mfenced></mrow></mfrac><mo linebreak="goodbreak">+</mo><msubsup><mi>AUC</mi><mi>t</mi><mrow><mi>C</mi><mo>/</mo><mi>D</mi></mrow></msubsup><mo>.</mo></mrow></math> </ephtml></p> <p>Thus, combining Equations (<reflink idref="bib3" id="ref41">3</reflink>) and (<reflink idref="bib4" id="ref42">4</reflink>), the <emph>C</emph>‐index can be derived using either incident or cumulative case definitions.</p> <p>In practice, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0076" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> is often observed to decrease monotonically with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0077" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi></mrow></math> </ephtml> (e.g., Figure 1). This behavior could, for example, be caused by a monotonically decreasing AUC curve, which tends to take smaller values as <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0078" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>t</mi></mrow></math> </ephtml> increases.[<reflink idref="bib26" id="ref43">26</reflink>] Note, however, that the monotonicity of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0079" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> does not hold in general and that it is possible to construct scenarios where <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0080" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> shows a distinctly non‐monotonic behavior (see Figure 2).</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01nov23/jrsm1655-fig-0002.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1655-fig-0002.jpg" title="2 Example of a concordance probability with non‐monotonic behavior. The black line (depicting the C‐index as a function τ) was derived by averaging 100 estimates of Cτ using the method of Uno et al.[14] Estimates were obtained from 100 independent samples with exponentially distributed event times (n=1000, rate = 1, no censoring). The true underlying model was given by η=−sin8⋅T2. The gray lines refer to the 100 sample‐specific curves. Although this example has been designed for illustrative purposes only and would be rarely encountered in practice, it shows that monotonicity of Cτ cannot be assumed in general." /> </p> <p></p> <hd id="AN0173455155-11">Estimation of the C ‐index</hd> <p>In the absence of censoring, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0086" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> is naturally estimated by the relative frequency <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0087" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>RF</mi></msub><mfenced open="(" close=")"><mi>τ</mi></mfenced><mo linebreak="goodbreak">=</mo><mfrac><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>i</mi><mo>≠</mo><mi>j</mi></mrow></munder><mspace width="0.1em" /><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>></mo><msub><mi>η</mi><mi>j</mi></msub></mrow></mfenced><mo>⋅</mo><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo><</mo><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>j</mi></msub></mrow></mfenced><mo>⋅</mo><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo>≤</mo><mi>τ</mi></mrow></mfenced></mrow><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>i</mi><mo>≠</mo><mi>j</mi></mrow></munder><mspace width="0.1em" /><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo><</mo><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>j</mi></msub></mrow></mfenced><mo>⋅</mo><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo>≤</mo><mi>τ</mi></mrow></mfenced></mrow></mfrac><mo>,</mo></mrow></math> </ephtml> which compares the orderings of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0088" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0089" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>η</mi><mi>i</mi></msub></mrow></math> </ephtml> in an observation‐wise manner. When applied to right‐censored data, this approach is no longer appropriate, as pairs of observations where the shorter observed event time is censored ( <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0090" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo><</mo><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>j</mi></msub></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0091" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi mathvariant="normal">Δ</mi><mi>i</mi></msub><mo>=</mo><mn>0</mn></mrow></math> </ephtml> ) cannot be compared in a meaningful way. An obvious way to incorporate censoring is to discard all pairs of non‐comparable observations, yielding the estimator <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0092" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mtext>Harrell</mtext></msub><mfenced open="(" close=")"><mi>τ</mi></mfenced><mo linebreak="goodbreak">=</mo><mfrac><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>i</mi><mo>≠</mo><mi>j</mi></mrow></munder><mspace width="0.1em" /><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>></mo><msub><mi>η</mi><mi>j</mi></msub></mrow></mfenced><mo>⋅</mo><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo><</mo><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>j</mi></msub></mrow></mfenced><mo>⋅</mo><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo>≤</mo><mi>τ</mi></mrow></mfenced><mo>⋅</mo><msub><mi mathvariant="normal">Δ</mi><mi>i</mi></msub></mrow><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>i</mi><mo>≠</mo><mi>j</mi></mrow></munder><mspace width="0.1em" /><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo><</mo><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>j</mi></msub></mrow></mfenced><mo>⋅</mo><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo>≤</mo><mi>τ</mi></mrow></mfenced><mo>⋅</mo><msub><mi mathvariant="normal">Δ</mi><mi>i</mi></msub></mrow></mfrac><mo>.</mo></mrow></math> </ephtml></p> <p>During the past decades, this estimator (also termed "Harrell's <emph>C</emph>") has become the most popular way to evaluate <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0093" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> . However, it shows a notable upward bias if censoring rates are high.[[<reflink idref="bib12" id="ref44">12</reflink>], [<reflink idref="bib15" id="ref45">15</reflink>]] To address this issue, Uno et al. proposed an inverse‐probability‐of‐censoring‐weighted version of Harrell's <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0094" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi></mrow></math> </ephtml> (termed "Uno's <emph>C</emph>") that is defined by5 <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0095" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>Uno</mi></msub><mfenced open="(" close=")"><mi>τ</mi></mfenced><mo linebreak="goodbreak">=</mo><mfrac><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>i</mi><mo>≠</mo><mi>j</mi></mrow></munder><mspace width="0.1em" /><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>></mo><msub><mi>η</mi><mi>j</mi></msub></mrow></mfenced><mo>⋅</mo><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo><</mo><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>j</mi></msub></mrow></mfenced><mo>⋅</mo><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo>≤</mo><mi>τ</mi></mrow></mfenced><mo>⋅</mo><msub><mi mathvariant="normal">Δ</mi><mi>i</mi></msub><mo>/</mo><mover accent="true"><mi>G</mi><mo>̂</mo></mover><msup><mfenced open="(" close=")"><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub></mfenced><mn>2</mn></msup></mrow><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>i</mi><mo>≠</mo><mi>j</mi></mrow></munder><mspace width="0.1em" /><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo><</mo><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>j</mi></msub></mrow></mfenced><mo>⋅</mo><mi mathvariant="normal">I</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo>≤</mo><mi>τ</mi></mrow></mfenced><mo>⋅</mo><msub><mi mathvariant="normal">Δ</mi><mi>i</mi></msub><mo>/</mo><mover accent="true"><mi>G</mi><mo>̂</mo></mover><msup><mfenced open="(" close=")"><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub></mfenced><mn>2</mn></msup></mrow></mfrac><mo>,</mo></mrow></math> </ephtml> where <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0096" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mover accent="true"><mi>G</mi><mo>̂</mo></mover><mfenced open="(" close=")"><mo>⋅</mo></mfenced></mrow></math> </ephtml> is a consistent estimator of the censoring survival function <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0097" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>G</mi><mfenced open="(" close=")"><mi>t</mi></mfenced><mo>=</mo><mi mathvariant="normal">P</mi><mspace width="0.1em" /><mfenced open="(" close=")"><mrow><msub><mi>Z</mi><mi>i</mi></msub><mo>></mo><mi>t</mi></mrow></mfenced></mrow></math> </ephtml> obtained from the validation data.[<reflink idref="bib14" id="ref46">14</reflink>] Usually, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0098" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>G</mi><mfenced open="(" close=")"><mo>⋅</mo></mfenced></mrow></math> </ephtml> is estimated by the Kaplan–Meier method, although more complex models (e.g. depending on a set of covariates) might be considered. Assuming conditionally independent censoring (i.e. independence of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0099" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>T</mi><mi>i</mi></msub></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0100" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>Z</mi><mi>i</mi></msub></mrow></math> </ephtml><ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0101" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>∀</mo><mi>i</mi></mrow></math> </ephtml> given the covariates) and a correctly specified censoring model with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0102" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>G</mi><mfenced open="(" close=")"><mi>t</mi></mfenced><mo>></mo><mi>δ</mi><mo>></mo><mn>0</mn><mspace width="0.5em" /><mo>∀</mo><mi>t</mi></mrow></math> </ephtml> , Uno et al.[<reflink idref="bib14" id="ref47">14</reflink>] showed that <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0103" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>Uno</mi></msub><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> is weakly consistent for <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0104" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> as <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0105" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>n</mi><mo>→</mo><mi>∞</mi></mrow></math> </ephtml> .</p> <p> <emph>Remark</emph>: The estimator considered by Gerds et al. is slightly different from (<reflink idref="bib5" id="ref48">5</reflink>) in that is <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0106" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mover accent="true"><mi>G</mi><mo>̂</mo></mover><msup><mfenced open="(" close=")"><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub></mfenced><mn>2</mn></msup></mrow></math> </ephtml> replaced by <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0107" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mover accent="true"><mi>G</mi><mo>̂</mo></mover><mfenced open="(" close=")"><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub></mfenced><mo>⋅</mo><mover accent="true"><mi>G</mi><mo>̂</mo></mover><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo>−</mo></mrow></mfenced></mrow></math> </ephtml> in both the numerator and the denominator, where <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0108" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo>−</mo></mrow></math> </ephtml> refers to a time point that is infinitesimally smaller than <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0109" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub></mrow></math> </ephtml><subs>.</subs>[<reflink idref="bib12" id="ref49">12</reflink>] Clearly, this difference is only relevant when <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0110" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mover accent="true"><mi>G</mi><mo>̂</mo></mover><mfenced open="(" close=")"><mo>⋅</mo></mfenced></mrow></math> </ephtml> is not continuous in <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0111" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>t</mi></mrow></math> </ephtml> (for instance when the Kaplan–Meier method is used to estimate <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0112" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mover accent="true"><mi>G</mi><mo>̂</mo></mover><mfenced open="(" close=")"><mo>⋅</mo></mfenced></mrow></math> </ephtml> ). In our analysis we will use the R add‐on package <emph>pec</emph>[<reflink idref="bib27" id="ref50">27</reflink>] that implements the method by Gerds et al.[<reflink idref="bib12" id="ref51">12</reflink>] but refer to this estimator as "Uno's <emph>C</emph>".</p> <p>A major advantage of Harrell's <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0113" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi></mrow></math> </ephtml> and Uno's <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0114" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi></mrow></math> </ephtml> is that both estimators are non‐parametric in the sense that they do not make any assumptions on the distribution of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0115" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>T</mi></mrow></math> </ephtml> . An alternative way to deal with non‐comparable pairs of observations is to specify a parametric or semi‐parametric working model for <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0116" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>T</mi></mrow></math> </ephtml> (e.g., a Cox regression model) and to derive estimators of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0117" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> based on the characteristics of this model.[[<reflink idref="bib11" id="ref52">11</reflink>], [<reflink idref="bib13" id="ref53">13</reflink>], [<reflink idref="bib28" id="ref54">28</reflink>]] It is also possible to apply a model‐free estimator of the incident/dynamic AUC curve[<reflink idref="bib29" id="ref55">29</reflink>] and to estimate the <emph>C</emph>‐index via numerical integration of the AUC estimate. In this paper we will consider Harrell's <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0118" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi></mrow></math> </ephtml> and Uno's <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0119" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi></mrow></math> </ephtml> throughout.</p> <hd id="AN0173455155-12">On the role of the truncation time τ</hd> <p>As stated in Section 1, the unrestricted <emph>C</emph>‐index <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0121" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mspace width="0.1em" /><mi>P</mi><mspace width="0.1em" /><mfenced open="(" close=")" separators="|"><mrow><msub><mi>η</mi><mi>i</mi></msub><mo>></mo><msub><mi>η</mi><mi>j</mi></msub><mspace width="0.1em" /></mrow><mrow><mspace width="0.1em" /><msub><mi>T</mi><mi>i</mi></msub><mo><</mo><msub><mi>T</mi><mi>j</mi></msub></mrow></mfenced></mrow></math> </ephtml> comes with an intuitive probabilistic interpretation, comparing the rankings of the values <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0122" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>η</mi><mi>i</mi></msub></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0123" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>T</mi><mi>i</mi></msub></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0124" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mi>n</mi></mrow></math> </ephtml> . This interpretation is considerably less intuitive if an additional truncation time <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0125" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi><mo><</mo><mi>∞</mi></mrow></math> </ephtml> is included in the definition of the <emph>C</emph>‐index. Nonetheless, there exist both conceptual <emph>and</emph> technical reasons to prefer a restricted version of the concordance probability over the unrestricted one: First, the sample values <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0126" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mfenced open="(" close=")" separators=",,"><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><msub><mi mathvariant="normal">Δ</mi><mi>i</mi></msub><msub><mi>η</mi><mi>i</mi></msub></mfenced></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0127" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mi>n</mi></mrow></math> </ephtml> , are often obtained from a validation study with a limited follow‐up time. In this case, the maximum possible time horizon <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0128" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi></mrow></math> </ephtml> is naturally given by the length of the follow‐up time, implying that <emph>any</emph> estimate of the concordance probability derived from the validation data is a restricted one.[<reflink idref="bib30" id="ref56">30</reflink>] Second, the censoring model used in the definition of Uno's <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0129" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi></mrow></math> </ephtml> usually assumes <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0130" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>G</mi><mfenced open="(" close=")"><mi>t</mi></mfenced><mo>></mo><mi>δ</mi><mo>></mo><mn>0</mn><mspace width="0.5em" /><mo>∀</mo><mi>t</mi></mrow></math> </ephtml> , posing a problem if non‐ or semi‐parametric methods are applied to estimate <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0131" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>G</mi><mfenced open="(" close=")"><mo>⋅</mo></mfenced></mrow></math> </ephtml> beyond <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0132" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi><mo>≔</mo><msub><mi>max</mi><mi>i</mi></msub><mfenced open="(" close=")"><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub></mfenced></mrow></math> </ephtml> . In particular, the Kaplan–Meier estimator (being the predominant estimator of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0133" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>G</mi><mfenced open="(" close=")"><mo>⋅</mo></mfenced></mrow></math> </ephtml> in practice) is zero beyond <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0134" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>max</mi><mi>i</mi></msub><mfenced open="(" close=")"><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub></mfenced></mrow></math> </ephtml> if the longest observed event time corresponds to a censored observation (and does not even exist beyond <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0135" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>max</mi><mi>i</mi></msub><mfenced open="(" close=")"><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub></mfenced></mrow></math> </ephtml> if this observation has <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0136" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi mathvariant="normal">Δ</mi><mi>i</mi></msub><mo>=</mo><mn>1</mn></mrow></math> </ephtml> ). These problems can be avoided if a restricted version of the <emph>C</emph>‐index (with a suitably defined value of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0137" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi><mo><</mo><msub><mi>max</mi><mi>i</mi></msub><mfenced open="(" close=")"><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub></mfenced></mrow></math> </ephtml> ) is considered for analysis.</p> <hd id="AN0173455155-13">Meta‐regression of C ‐index estimates</hd> <p>In this section we describe a set of models to account for the time dependency of the restricted <emph>C</emph>‐index in meta‐regression. We start with the classical model for time‐independent meta‐analysis, also discussing possible transformations of <emph>C</emph>‐index estimates before model fitting.</p> <p> <emph>Random‐effects meta‐analysis</emph>. Consider a set of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0138" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi></mrow></math> </ephtml> independent validation studies with reported study‐specific estimates <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0139" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mn>1</mn></msub><mo>,</mo><mi>...</mi><mo>,</mo><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>K</mi></msub></mrow></math> </ephtml> and reported variance estimates <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0140" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mover accent="true"><mi>σ</mi><mo>̂</mo></mover><mn>1</mn><mn>2</mn></msubsup><mo>,</mo><mi>...</mi><mo>,</mo><msubsup><mover accent="true"><mi>σ</mi><mo>̂</mo></mover><mi>K</mi><mn>2</mn></msubsup></mrow></math> </ephtml> . As argued above, each of these estimates relates to a study‐specific truncation time <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0141" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0142" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>k</mi><mo>=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mi>K</mi></mrow></math> </ephtml> . Classical parametric meta‐analysis ignores this time dependency, assuming that <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0143" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mn>1</mn></msub><mo>,</mo><mi>...</mi><mo>,</mo><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>K</mi></msub></mrow></math> </ephtml> are estimates of some study‐specific unrestricted concordance probabilities <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0144" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>C</mi><mn>1</mn></msub><mo>,</mo><mi>...</mi><mo>,</mo><msub><mi>C</mi><mi>K</mi></msub></mrow></math> </ephtml> . We further assume (here and in all other models, following standard procedures) that each <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0145" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mover accent="true"><mi>σ</mi><mo>̂</mo></mover><mi>k</mi><mn>2</mn></msubsup></mrow></math> </ephtml> corresponds to the true variance <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0146" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>σ</mi><mi>k</mi><mn>2</mn></msubsup></mrow></math> </ephtml> of the respective residual term <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0147" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi mathvariant="normal">ϵ</mi><mi>k</mi></msub><mo>=</mo><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub><mo>−</mo><msub><mi>C</mi><mi>k</mi></msub></mrow></math> </ephtml> . The corresponding model is given by <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0148" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub><mo>=</mo><msub><mi>C</mi><mi>k</mi></msub><mo>+</mo><msub><mi mathvariant="normal">ϵ</mi><mi>k</mi></msub><mo>,</mo><mo> </mo><msub><mi mathvariant="normal">ϵ</mi><mi>k</mi></msub><mo>∼</mo><mi>N</mi><mfenced open="(" close=")" separators=","><mrow><mn>0</mn><mo>,</mo><msubsup><mi>σ</mi><mi>k</mi><mn>2</mn></msubsup></mrow></mfenced><mo>,</mo></mrow></math> </ephtml> 6 <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0149" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>C</mi><mi>k</mi></msub><mo>=</mo><msub><mi>C</mi><mi>pop</mi></msub><mo>+</mo><msub><mi>a</mi><mi>k</mi></msub><mo>,</mo><msub><mi>a</mi><mi>k</mi></msub><mo>∼</mo><mi>N</mi><mfenced open="(" close=")" separators=","><mrow><mn>0</mn><mo>,</mo><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup></mrow></mfenced><mo>,</mo><mo> </mo><mi>k</mi><mo>=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mi>K</mi><mo>,</mo></mrow></math> </ephtml> where the aim is to obtain a "pooled" estimate of the population value <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0150" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>C</mi><mi>pop</mi></msub></mrow></math> </ephtml> . The study‐specific deviations <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0151" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>a</mi><mi>k</mi></msub></mrow></math> </ephtml> are assumed to be independent of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0152" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi mathvariant="normal">ϵ</mi><mi>k</mi></msub></mrow></math> </ephtml> and to follow a normal distribution with between‐study variance <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0153" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup></mrow></math> </ephtml> .</p> <p>If homogeneity of studies is assumed, i.e. <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0154" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup><mo>=</mo><mn>0</mn></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0155" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>C</mi><mn>1</mn></msub><mo>=</mo><mi>...</mi><mo>=</mo><msub><mi>C</mi><mi>k</mi></msub><mo>=</mo><msub><mi>C</mi><mi>pop</mi></msub></mrow></math> </ephtml> , then this is referred to as common‐effect meta‐analysis. In contrast, random‐effects meta‐analysis assumes <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0156" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup><mo>≠</mo><mn>0</mn></mrow></math> </ephtml> , accounting for study‐specific heterogeneity. Results of validation studies are usually expected to vary between studies, as these may differ in sample selection and many other design aspects. Therefore, and in line with the recommendation of Debray et al.,[<reflink idref="bib6" id="ref57">6</reflink>] we will restrict our analysis to random‐effects models for the purpose of our study. In the literature, numerous methods to estimate <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0157" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup></mrow></math> </ephtml> have been proposed.[<reflink idref="bib31" id="ref58">31</reflink>] Here we follow the recommendation by Debray et al.[<reflink idref="bib6" id="ref59">6</reflink>] and consider methods based on restricted maximum likelihood (REML) estimation. With this approach, estimation of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0158" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0159" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>C</mi><mi>pop</mi></msub></mrow></math> </ephtml> is performed jointly using a model with inverse variance weights <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0160" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>1</mn><mo>/</mo><msubsup><mover accent="true"><mi>σ</mi><mo>̂</mo></mover><mi>k</mi><mn>2</mn></msubsup></mrow></math> </ephtml> .</p> <p> <emph>Transformations of C</emph>‐<emph>index estimates</emph>. The classical approach to meta‐analyze <emph>C</emph>‐index values is based on the untransformed estimates <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0161" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mn>1</mn></msub><mo>,</mo><mi>...</mi><mo>,</mo><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>K</mi></msub></mrow></math> </ephtml> . This approach, which relies on the asymptotic normality of estimators like Uno's <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0162" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi></mrow></math> </ephtml> , has been followed e.g. by Büttner et al.[<reflink idref="bib20" id="ref60">20</reflink>] and Waldron et al.[<reflink idref="bib32" id="ref61">32</reflink>] Other authors have argued that the concordance probability is bounded between 0 and 1, so that the normality and homoscedasticity assumptions in (<reflink idref="bib6" id="ref62">6</reflink>) are unlikely to hold. To address these issues, they transformed <emph>C</emph>‐index estimates before meta‐analysis, using e.g. the logistic transformation <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0163" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi><mfenced open="(" close=")"><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub></mfenced><mo>=</mo><mi>log</mi><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub><mo>/</mo><mfenced open="(" close=")"><mrow><mn>1</mn><mo>−</mo><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub></mrow></mfenced></mrow></mfenced></mrow></math> </ephtml> [<reflink idref="bib33" id="ref63">33</reflink>] or the arcsine square root transformation <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0164" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi><mfenced open="(" close=")"><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub></mfenced><mo>=</mo><msup><mi>sin</mi><mrow><mo>−</mo><mn>1</mn></mrow></msup><mfenced open="(" close=")"><msubsup><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi><mrow><mn>1</mn><mo>/</mo><mn>2</mn></mrow></msubsup></mfenced></mrow></math> </ephtml><subs>.</subs>[<reflink idref="bib34" id="ref64">34</reflink>] After model fitting, the estimate of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0165" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>C</mi><mi>pop</mi></msub></mrow></math> </ephtml> is usually back‐transformed to the original probability scale.</p> <p> <emph>Linear meta‐regression</emph>. As argued above, classical meta‐analysis does not account for the implicit time dependency of the estimates <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0166" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mn>1</mn></msub><mo>,</mo><mi>...</mi><mo>,</mo><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>K</mi></msub></mrow></math> </ephtml> . As a consequence, it is unclear how to interpret the population value <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0167" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>C</mi><mi>pop</mi></msub></mrow></math> </ephtml> in Equation (<reflink idref="bib6" id="ref65">6</reflink>). In particular, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0168" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>C</mi><mi>pop</mi></msub></mrow></math> </ephtml> will not be a meaningful approximation of the unrestricted <emph>C</emph>‐index if <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0169" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> decreases with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0170" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi></mrow></math> </ephtml> (see Figure 1).</p> <p>A more appropriate approach to account for the time dependency of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0171" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> is to consider a meta‐regression model of the form7 <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0172" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi><mfenced open="(" close=")"><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub></mfenced><mo linebreak="goodbreak">=</mo><mi>f</mi><mfenced open="(" close=")" separators=";"><msub><mi>τ</mi><mi>k</mi></msub><mi>γ</mi></mfenced><mo linebreak="goodbreak">+</mo><msub><mi>a</mi><mi>k</mi></msub><mo linebreak="goodbreak">+</mo><msub><mi mathvariant="normal">ϵ</mi><mi>k</mi></msub><mo>,</mo><mspace width="1em" /><msub><mi>a</mi><mi>k</mi></msub><mo>∼</mo><mi>N</mi><mfenced open="(" close=")" separators=","><mn>0</mn><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup></mfenced><mo>,</mo><mspace width="1em" /><msub><mi mathvariant="normal">ϵ</mi><mi>k</mi></msub><mo>∼</mo><mi>N</mi><mfenced open="(" close=")" separators=","><mn>0</mn><msubsup><mi>σ</mi><mi>k</mi><mn>2</mn></msubsup></mfenced><mo>,</mo></mrow></math> </ephtml><ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0173" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>k</mi><mo>=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mi>K</mi></mrow></math> </ephtml> , where <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0174" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi><mfenced open="(" close=")"><mo>⋅</mo></mfenced></mrow></math> </ephtml> is a pre‐specified transformation (for instance, the logistic transformation) and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0175" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> is included as a covariate. The relationship between <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0176" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0177" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> is modeled by the (possibly nonlinear) function <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0178" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>f</mi><mfenced open="(" close=")"><mo>⋅</mo></mfenced></mrow></math> </ephtml> depending on a coefficient vector <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0179" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>γ</mi><mo>∈</mo><msup><mi mathvariant="normal">ℝ</mi><mi>p</mi></msup></mrow></math> </ephtml> . Instead of calculating a one‐dimensional pooled estimate of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0180" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>C</mi><mi>pop</mi></msub></mrow></math> </ephtml> , the idea is to first estimate the coefficient vector <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0181" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>γ</mi></mrow></math> </ephtml> and to subsequently approximate the full curve <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0182" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> by the estimated function <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0183" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msup><mi>g</mi><mrow><mo>−</mo><mn>1</mn></mrow></msup><mfenced open="(" close=")"><mrow><mi>f</mi><mfenced open="(" close=")" separators=";"><mi>τ</mi><mover accent="true"><mi>γ</mi><mo>̂</mo></mover></mfenced></mrow></mfenced></mrow></math> </ephtml> .</p> <p>The simplest way of specifying a model of the form (<reflink idref="bib7" id="ref66">7</reflink>) is to consider the linear function <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0184" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>f</mi><mfenced open="(" close=")" separators=";"><msub><mi>τ</mi><mi>k</mi></msub><mi>γ</mi></mfenced><mo>=</mo><msub><mi>γ</mi><mn>0</mn></msub><mo>+</mo><msub><mi>τ</mi><mi>k</mi></msub><mo>⋅</mo><msub><mi>γ</mi><mn>1</mn></msub></mrow></math> </ephtml> , yielding the <emph>linear meta‐regression model</emph> with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0185" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>γ</mi><mo>=</mo><msup><mfenced open="(" close=")" separators=","><msub><mi>γ</mi><mn>0</mn></msub><msub><mi>γ</mi><mn>1</mn></msub></mfenced><mi>Τ</mi></msup><mo>∈</mo><msup><mi mathvariant="normal">ℝ</mi><mn>2</mn></msup></mrow></math> </ephtml> . Estimation of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0186" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>γ</mi></mrow></math> </ephtml> is performed in the same way as above, i.e. using REML with inverse variance weights <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0187" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>1</mn><mo>/</mo><msubsup><mover accent="true"><mi>σ</mi><mo>̂</mo></mover><mi>k</mi><mn>2</mn></msubsup></mrow></math> </ephtml> .</p> <p> <emph>Spline meta‐regression</emph>. Although the linear meta‐regression model accounts for the time dependency of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0188" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> , it does not capture nonlinear functional relationships as the ones presented in Figures 1 and 2. This might be a problem even when the values of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0189" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub></mrow></math> </ephtml> are transformed before model fitting. A convenient approach to address nonlinearity is to represent <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0190" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>f</mi><mfenced open="(" close=")" separators=";"><msub><mi>τ</mi><mi>k</mi></msub><mi>γ</mi></mfenced></mrow></math> </ephtml> by a restricted cubic spline, as implemented in the R packages <emph>metafor</emph> and <emph>rms</emph>.[[<reflink idref="bib35" id="ref67">35</reflink>]] With this approach, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0191" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>f</mi><mfenced open="(" close=")" separators=";"><msub><mi>τ</mi><mi>k</mi></msub><mi>γ</mi></mfenced></mrow></math> </ephtml> is specified as a weighted sum of truncated power basis functions (defined using a pre‐specified set of <emph>interior knots</emph>), and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0192" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>γ</mi></mrow></math> </ephtml> is set equal to the vector of weights. Regarding the number and placement of the knots, we follow the recommendations in section 2.4.6 of Harrell Jr.,[<reflink idref="bib37" id="ref68">37</reflink>] using four knots (= basis functions) if <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0193" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>≥</mo><mn>30</mn></mrow></math> </ephtml> and three knots if <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0194" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo><</mo><mn>30</mn></mrow></math> </ephtml> . Obviously, the spline meta‐regression model depends on a larger number of coefficients than the linear meta‐regression model, increasing its flexibility but also being more prone to overfitting (especially when the number of studies is small).</p> <p> <emph>Fractional polynomial meta‐regression</emph>. An alternative to spline regression is fractional polynomial (FP) modeling, which is based on transformations of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0195" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> by a weighted sum of power functions. Following Royston and Sauerbrei,[<reflink idref="bib38" id="ref69">38</reflink>] we consider fractional polynomials of degree 2 ("FP2"), which are defined by <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0196" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>f</mi><mfenced open="(" close=")" separators=";"><msub><mi>τ</mi><mi>k</mi></msub><mi>γ</mi></mfenced><mo>=</mo><msub><mi>γ</mi><mn>0</mn></msub><mo>+</mo><msub><mi>γ</mi><mn>1</mn></msub><mo>⋅</mo><msubsup><mi>τ</mi><mi>k</mi><msub><mi>p</mi><mn>1</mn></msub></msubsup><mo>+</mo><msub><mi>γ</mi><mn>2</mn></msub><mo>⋅</mo><msubsup><mi>τ</mi><mi>k</mi><msub><mi>p</mi><mn>2</mn></msub></msubsup></mrow></math> </ephtml> , where <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0197" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>p</mi><mn>1</mn></msub></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0198" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>p</mi><mn>2</mn></msub></mrow></math> </ephtml> are chosen from the predefined set of powers <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0199" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>S</mi><mo>=</mo><mfenced open="{" close="}"><mrow><mo>−</mo><mn>2</mn><mo>,</mo><mo>−</mo><mn>1</mn><mo>,</mo><mo>−</mo><mn>0.5,0,0.5,1,2,3</mn></mrow></mfenced></mrow></math> </ephtml> with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0200" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>τ</mi><mi>k</mi><mn>0</mn></msubsup><mo>≔</mo><mi>log</mi><mfenced open="(" close=")"><msub><mi>τ</mi><mi>k</mi></msub></mfenced></mrow></math> </ephtml> . In case <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0201" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>p</mi><mn>1</mn></msub><mo>=</mo><msub><mi>p</mi><mn>2</mn></msub><mo>≕</mo><msup><mi>p</mi><mo>*</mo></msup></mrow></math> </ephtml> , the function <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0202" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>f</mi><mfenced open="(" close=")"><mo>⋅</mo></mfenced></mrow></math> </ephtml> is defined by <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0203" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>f</mi><mfenced open="(" close=")" separators=";"><msub><mi>τ</mi><mi>k</mi></msub><mi>γ</mi></mfenced><mo>=</mo><msub><mi>γ</mi><mn>0</mn></msub><mo>+</mo><msub><mi>γ</mi><mn>1</mn></msub><mo>⋅</mo><msubsup><mi>τ</mi><mi>k</mi><msup><mi>p</mi><mo>*</mo></msup></msubsup><mo>+</mo><msub><mi>γ</mi><mn>2</mn></msub><mo>⋅</mo><msubsup><mi>τ</mi><mi>k</mi><msup><mi>p</mi><mo>*</mo></msup></msubsup><mo>⋅</mo><mi>log</mi><mfenced open="(" close=")"><msub><mi>τ</mi><mi>k</mi></msub></mfenced></mrow></math> </ephtml> . As demonstrated by Royston and Sauerbrei[<reflink idref="bib38" id="ref70">38</reflink>] and Royston and Altman,[<reflink idref="bib39" id="ref71">39</reflink>] FP2 models are able to capture a wide variety of nonlinear trends, providing "enough flexibility for modelling many of the types of continuous function we encounter in the health sciences and elsewhere" (Royston & Sauerbrei,[<reflink idref="bib38" id="ref72">38</reflink>] p. 73). For meta‐regression of the <emph>C</emph>‐index we propose to use a function of the form <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0204" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>f</mi><mfenced open="(" close=")" separators=";"><msub><mi>τ</mi><mi>k</mi></msub><mi>γ</mi></mfenced><mo>=</mo><msub><mi>γ</mi><mn>0</mn></msub><mo>+</mo><msub><mi>γ</mi><mn>1</mn></msub><mo>⋅</mo><msubsup><mi>τ</mi><mi>k</mi><mrow><mo>−</mo><mn>0.5</mn></mrow></msubsup><mo>+</mo><msub><mi>γ</mi><mn>2</mn></msub><mo>⋅</mo><msubsup><mi>τ</mi><mi>k</mi><mn>0.5</mn></msubsup></mrow></math> </ephtml> , i.e. <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0205" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>p</mi><mn>1</mn></msub></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0206" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>p</mi><mn>2</mn></msub></mrow></math> </ephtml> are set to <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0207" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>−</mo><mn>0.5</mn></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0208" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>0.5</mn></mrow></math> </ephtml> , respectively. The latter values are inspired by the "typical" shape of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0209" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> in Figure 1 and by the fact that this shape closely resembles the respective FP2 plot in Figure 1 of Royston.[<reflink idref="bib40" id="ref73">40</reflink>] Section 4 presents a detailed empirical analysis of the choice of the power values.</p> <p> <emph>Exponential decay meta‐regression</emph>. In addition to the aforementioned meta‐regression models, we consider the <emph>exponential decay meta‐regression model</emph>, which employs an alternative regression function that requires the time‐restricted concordance index to be monotone decreasing with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0210" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi></mrow></math> </ephtml> . This approach might be suitable when there is strong evidence of a monotonic trend in <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0211" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> (as the one shown in Figure 1). The exponential decay meta‐regression model is specified as8 <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0212" xmlns="http://www.w3.org/1998/Math/MathML"><mi>g</mi><mfenced open="(" close=")"><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub></mfenced><mo linebreak="goodbreak">=</mo><mi>θ</mi><mo linebreak="goodbreak">+</mo><msub><mi>a</mi><mi>k</mi></msub><mo linebreak="goodbreak">+</mo><mfenced open="(" close=")"><mrow><msub><mi>R</mi><mn>0</mn></msub><mo linebreak="goodbreak">−</mo><mfenced open="(" close=")"><mrow><mi>θ</mi><mo linebreak="goodbreak">+</mo><msub><mi>a</mi><mi>k</mi></msub></mrow></mfenced></mrow></mfenced><mo>⋅</mo><mi>exp</mi><mfenced open="(" close=")"><mrow><mo linebreak="goodbreak">−</mo><mi>exp</mi><mfenced open="(" close=")"><mi>β</mi></mfenced><mo>⋅</mo><msub><mi>τ</mi><mi>k</mi></msub></mrow></mfenced><mspace linebreak="newline" /><mspace width="6em" /><mo linebreak="goodbreak">+</mo><msub><mi mathvariant="normal">ϵ</mi><mi>k</mi></msub><mo>,</mo><mspace width="0.5em" /><msub><mi>a</mi><mi>k</mi></msub><mo>∼</mo><mi>N</mi><mfenced open="(" close=")" separators=","><mn>0</mn><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup></mfenced><mo>,</mo><mspace width="1em" /><msub><mi mathvariant="normal">ϵ</mi><mi>k</mi></msub><mo>∼</mo><mi>N</mi><mfenced open="(" close=")" separators=","><mn>0</mn><msubsup><mi>σ</mi><mi>k</mi><mn>2</mn></msubsup></mfenced><mo>,</mo></math> </ephtml> with parameter vector <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0213" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>γ</mi><mo>=</mo><msup><mfenced open="(" close=")" separators=",,"><mi>θ</mi><mi>β</mi><msub><mi>R</mi><mn>0</mn></msub></mfenced><mi>T</mi></msup></mrow></math> </ephtml> . By definition, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0214" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>g</mi><mfenced open="(" close=")"><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub></mfenced></mrow></math> </ephtml> converges to <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0215" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>θ</mi><mo>+</mo><msub><mi>a</mi><mi>k</mi></msub><mo>+</mo><msub><mi mathvariant="normal">ϵ</mi><mi>k</mi></msub></mrow></math> </ephtml> as <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0216" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub><mo>→</mo><mi>∞</mi></mrow></math> </ephtml> , implying that the unrestricted <emph>C</emph>‐index might be estimated by the fitted value of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0217" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>θ</mi></mrow></math> </ephtml> . (Note that this is not possible with the linear, spline, and fractional polynomial regression approaches described above.) The value of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0218" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>R</mi><mn>0</mn></msub></mrow></math> </ephtml> corresponds to an approximation of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0219" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><msub><mi>τ</mi><mi>k</mi></msub></mfenced></mrow></math> </ephtml> at <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0220" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub><mo>=</mo><mn>0</mn></mrow></math> </ephtml> , and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0221" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>β</mi></mrow></math> </ephtml> determines the rate of decay. Further note that the random‐effects structure of the exponential decay model is slightly different from the respective structure in (<reflink idref="bib7" id="ref74">7</reflink>), as the random effect <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0222" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>a</mi><mi>k</mi></msub></mrow></math> </ephtml> enters (<reflink idref="bib8" id="ref75">8</reflink>) in a nonlinear way.</p> <hd id="AN0173455155-14">SIMULATION STUDY</hd> <p></p> <hd id="AN0173455155-15">Experimental setup</hd> <p>Here we present the results of a simulation study that we conducted to analyze the properties of the models discussed in Section 2. The aims of our study were (i) to investigate the benefit of incorporating the truncation times <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0223" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> in meta‐regression models for the concordance probability, (ii) to compare the performance of the meta‐regression approaches discussed in Section 2 with regard to estimation accuracy and numerical stability, and (iii) to investigate the use of variable transformations before model fitting.</p> <p>Our simulation study was based on a Weibull model of the form9 <ephtml> <math display="block" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0224" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>log</mi><mfenced open="(" close=")"><msub><mi>T</mi><mi>i</mi></msub></mfenced><mo linebreak="goodbreak">=</mo><msub><mi>η</mi><mi>i</mi></msub><mo linebreak="goodbreak">−</mo><mi>σ</mi><msub><mi>W</mi><mi>i</mi></msub><mo>,</mo><mspace width="1em" /><msub><mi>η</mi><mi>i</mi></msub><mo>∼</mo><mi>N</mi><mfenced open="(" close=")"><msup><mn>0,0.5</mn><mn>2</mn></msup></mfenced><mo>,</mo><mspace width="1em" /><mi>i</mi><mo linebreak="goodbreak">=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mi>n</mi><mo>,</mo></mrow></math> </ephtml> with normally distributed score values <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0225" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>η</mi><mi>i</mi></msub></mrow></math> </ephtml> and noise variables <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0226" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>W</mi><mi>i</mi></msub></mrow></math> </ephtml> that followed a standard Gumbel distribution (independent of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0227" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>η</mi><mi>i</mi></msub></mrow></math> </ephtml> ). The parameter <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0228" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>σ</mi></mrow></math> </ephtml> was set to 0.5, yielding the <emph>C</emph>‐index curve presented in Figure S1. For example, we obtained <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0229" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><msub><mi>τ</mi><mi>k</mi></msub></mfenced><mo>=</mo><mn>0.79</mn></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0230" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>0.77</mn></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0231" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>0.74</mn></mrow></math> </ephtml> for <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0232" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub><mo>=</mo><mn>0.2</mn></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0233" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>0.7</mn></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0234" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>1.5</mn></mrow></math> </ephtml> , respectively.</p> <p>Based on Model (<reflink idref="bib9" id="ref76">9</reflink>), we considered four scenarios for meta‐regression, setting the number of validation studies to <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0235" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>10</mn></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0236" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>15</mn></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0237" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>30</mn></mrow></math> </ephtml> , and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0238" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>50</mn></mrow></math> </ephtml> . For each <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0239" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi></mrow></math> </ephtml> we simulated study data sets with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0240" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>n</mi><mi>k</mi></msub></mrow></math> </ephtml> observations ( <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0241" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>k</mi><mo>=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mi>K</mi></mrow></math> </ephtml> ), generating the sample sizes <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0242" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>n</mi><mi>k</mi></msub></mrow></math> </ephtml> randomly from the grid <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0243" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mfenced open="{" close="}"><mrow><mn>100,110,120</mn><mo>,</mo><mi>...</mi><mn>,990,1000</mn></mrow></mfenced></mrow></math> </ephtml> . The censoring times <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0244" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>Z</mi><mi>i</mi></msub></mrow></math> </ephtml> were sampled from an exponential distribution with rate parameter <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0245" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>0.5</mn></mrow></math> </ephtml> . The truncation times <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0246" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> of the studies were generated as follows: First, we defined a joint maximum follow‐up time (denoted by <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0247" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>max</mi></msub></mrow></math> </ephtml> ) for all studies. Afterwards we sampled the values <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0248" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> from a truncated gamma distribution on the interval <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0249" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mfenced open="[" close="]" separators=";"><mn>0.1</mn><msub><mi>τ</mi><mi>max</mi></msub></mfenced></mrow></math> </ephtml> . The shape and rate parameters of this distribution were set to 1.5 and 1, respectively. Subsequently, event times with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0250" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>T</mi><mo>˜</mo></mover><mi>i</mi></msub><mo>></mo><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> were censored at <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0251" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> (study‐wise). We considered three values of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0252" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>max</mi></msub></mrow></math> </ephtml> , namely <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0253" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>max</mi></msub><mo>=</mo><mn>0.7</mn></mrow></math> </ephtml> ("short follow‐up"), <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0254" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>max</mi></msub><mo>=</mo><mn>0.9</mn></mrow></math> </ephtml> ("medium follow‐up"), and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0255" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>max</mi></msub><mo>=</mo><mn>2</mn></mrow></math> </ephtml> ("long follow‐up"), yielding average censoring rates of 0.92, 0.86 and 0.64, respectively. Estimates of the <emph>C</emph>‐index were obtained using Uno's <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0256" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi></mrow></math> </ephtml> , as implemented in the R package <bold>pec</bold>. To introduce study‐specific heterogeneity, we added normally distributed random numbers <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0257" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>a</mi><mi>k</mi></msub></mrow></math> </ephtml> to the <emph>C</emph>‐index estimates. These numbers were drawn from a normal distribution with zero mean and variance <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0258" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup></mrow></math> </ephtml> . Again we considered three values, setting <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0259" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup><mo>=</mo><mn>0</mn></mrow></math> </ephtml> ("no heterogeneity"), <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0260" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup><mo>=</mo><msup><mn>0.01</mn><mn>2</mn></msup></mrow></math> </ephtml> ("moderate heterogeneity"), and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0261" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup><mo>=</mo><msup><mn>0.03</mn><mn>2</mn></msup></mrow></math> </ephtml> ("large heterogeneity"). The choice of these numbers was inspired by our work in Zacharias et al.,[<reflink idref="bib25" id="ref77">25</reflink>] where differences in <emph>C</emph>‐index values varied between 0.002 and 0.06 across validation studies (the latter number corresponding to two standard deviations of our "large heterogeneity" setting).</p> <p>In each of the <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0262" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>4</mn><mo>×</mo><mn>3</mn><mo>×</mo><mn>3</mn><mo>=</mo><mn>36</mn></mrow></math> </ephtml> scenarios (defined by the values of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0263" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0264" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>max</mi></msub></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0265" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>σ</mi><mi>a</mi><mn>2</mn></msubsup></mrow></math> </ephtml> ) we set the number of Monte Carlo replications to 1000 and fitted the following models to the simulated study data: (i) meta‐analysis, (ii) linear meta‐regression, (iii) spline meta‐regression, (iv) fractional polynomial meta‐regression, and (v) exponential decay meta‐regression, as described in Section 2. Model fitting was carried out using the metamean, metareg and rma functions of the R packages <emph>meta</emph> and <emph>metafor</emph>,[[<reflink idref="bib35" id="ref78">35</reflink>], [<reflink idref="bib41" id="ref79">41</reflink>]] except for the exponential decay model for which the function nlme of the R package <emph>nlme</emph>[<reflink idref="bib42" id="ref80">42</reflink>] was used. For sensitivity analysis, we additionally carried out random‐effects meta‐analyses using the <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0266" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>30</mn><mo>%</mo></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0267" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>50</mn><mo>%</mo></mrow></math> </ephtml> of studies with largest values of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0268" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> only. This approach was inspired by the shape of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0269" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> in Figure 1, assuming that studies with a long follow‐up time would be less affected by time dependency due to the convergence behavior of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0270" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> . Standard errors of the <emph>C</emph>‐index estimates (needed to calculate the weights <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0271" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>1</mn><mo>/</mo><msubsup><mover accent="true"><mi>σ</mi><mo>̂</mo></mover><mi>k</mi><mn>2</mn></msubsup></mrow></math> </ephtml> ) were computed using <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0272" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>1000</mn></mrow></math> </ephtml> bootstrap samples with replacement.</p> <p>Transformation functions included the identity transformation (id), the logistic transformation (logit), and the arcsine square root transformation (asin). Another candidate transformation would have been the double arcsine transformation; however, we did not consider this transformation because it has recently been found unsuitable for meta‐analysis purposes.[<reflink idref="bib43" id="ref81">43</reflink>] Comparisons of the respective <emph>C</emph>‐index estimates were carried out at the population level (setting <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0273" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>σ</mi><mi>a</mi></msub><mo>=</mo><mn>0</mn></mrow></math> </ephtml> ) after back transforming the fitted values to the original scale.</p> <p>The following criteria were used to evaluate the results of the simulation study:</p> <p></p> <ulist> <item> To investigate the numerical stability of the methods, we calculated the proportion of simulation runs in which the respective R fitting functions issued errors and/or warnings indicating convergence issues. These assessments were necessary since each of the studies entered the models with a separate random effect <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0274" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>a</mi><mi>k</mi></msub></mrow></math> </ephtml> and a separate variance term <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0275" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>σ</mi><mi>k</mi><mn>2</mn></msubsup></mrow></math> </ephtml> , potentially leading to some instabilities in the REML procedure.</item> <p></p> <item> To investigate the estimation accuracy of the meta‐regression models at a fixed time point, we evaluated the pooled estimates of the restricted concordance probability at <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0276" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>t</mi><mo>=</mo><mn>0.8</mn><mo>⋅</mo><msub><mi>τ</mi><mi>max</mi></msub></mrow></math> </ephtml> and compared these estimates (including their <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0277" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>95</mn><mo>%</mo></mrow></math> </ephtml> confidence intervals) to the respective true values of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0278" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mrow><mn>0.8</mn><mo>⋅</mo><msub><mi>τ</mi><mi>max</mi></msub></mrow></mfenced></mrow></math> </ephtml> .</item> <p></p> <item> For all methods we computed the areas enclosed by the true and the estimated <emph>C</emph> ‐index curves, using <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0279" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>min</mi><mi>k</mi></msub><mfenced open="(" close=")"><msub><mi>τ</mi><mi>k</mi></msub></mfenced></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0280" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>max</mi><mi>k</mi></msub><mfenced open="(" close=")"><msub><mi>τ</mi><mi>k</mi></msub></mfenced></mrow></math> </ephtml> as interval limits. All areas were divided by the interval length <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0281" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mfenced open="(" close=")"><mrow><msub><mi>max</mi><mi>k</mi></msub><mfenced open="(" close=")"><msub><mi>τ</mi><mi>k</mi></msub></mfenced><mo>−</mo><msub><mi>min</mi><mi>k</mi></msub><mfenced open="(" close=")"><msub><mi>τ</mi><mi>k</mi></msub></mfenced></mrow></mfenced></mrow></math> </ephtml> , see Figure S2 for an illustration.</item> </ulist> <hd id="AN0173455155-16">Results</hd> <p>Tables S1–S4 contain a summary of the failure rates that is, the percentages of the simulation runs in which the R fitting functions issued either an error or a warning. It is seen that fitting the exponential decay model resulted in a large number of convergence issues. For example, in the scenario with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0282" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>30</mn></mrow></math> </ephtml> studies (Table S3), failure rates were as high as <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0283" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>74.3</mn><mo>%</mo></mrow></math> </ephtml> of the simulation runs. Generally, failure rates tended to decrease with the length of follow‐up, which can be explained by the more pronounced curvature of the <emph>C</emph>‐index curve in these scenarios (showing stronger support for the shape of the exponential decay function). Still, failure rates were high even in the most favorable settings. We conclude that the exponential decay method may not be recommended for meta‐regression of the concordance index, and we therefore did not consider this model further. The failure rates of the other methods were throughout close to zero.</p> <p>In the remainder of this section, we present the results obtained from the scenario with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0284" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>30</mn></mrow></math> </ephtml> studies. The results of the other three scenarios ( <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0285" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>10</mn></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0286" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>15</mn></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0287" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>50</mn></mrow></math> </ephtml> ) are presented in the Supporting Information.</p> <p>Figure 3 presents the pooled concordance probability estimates at the fixed truncation time <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0288" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>0.8</mn><mo>⋅</mo><msub><mi>τ</mi><mi>max</mi></msub></mrow></math> </ephtml> (logistic transformation, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0289" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>30</mn></mrow></math> </ephtml> ). It is seen that ignoring the time‐dependency of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0290" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> resulted in a bias of classical random‐effects meta‐analysis. In line with Figure 1, this bias was positive in most of the scenarios and was most pronounced when the follow‐up time was long. It was close to zero on average when the follow‐up time was short. As expected, the estimates obtained from the sensitivity analyses (corresponding to random‐effects analyses of the <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0291" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>30</mn><mo>%</mo></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0292" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>50</mn><mo>%</mo></mrow></math> </ephtml> of studies with largest values of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0293" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> ) were almost unbiased in the scenarios with long follow‐up. The meta‐regression methods performed well in all settings, with spline meta‐regression showing a higher variability than linear and fractional polynomial meta‐regression. As expected, the variance of the estimates increased as the heterogeneity between studies became larger.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01nov23/jrsm1655-fig-0003.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1655-fig-0003.jpg" title="3 Results of the simulation study (K=30). The boxplots summarize the pooled estimates of the restricted concordance index at 0.8⋅τmax. All C‐index estimates were transformed using a logistic transformation before model fitting. The red and the gray lines refer to the true values of C0.8⋅τmax and the unrestricted values of the concordance index, respectively. Note that the gray lines coincide with the red lines in the lower three panels. [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <p>Similar results were obtained in the scenarios with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0297" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>10</mn></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0298" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>15</mn></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0299" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>50</mn></mrow></math> </ephtml> (Figures S3–S5, respectively). The results obtained from the untransformed and arcsine‐square‐root‐transformed estimates ( <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0300" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>30</mn></mrow></math> </ephtml> ) are presented in Figures S6 and S7, respectively. Compared to the logit‐transformed estimates, these estimates showed a slightly increased bias, especially in the scenarios with short follow‐up. Again, spline meta‐regression had a higher variability than fractional polynomial meta‐regression.</p> <p>Figure 4 presents the estimated coverage probabilities of the <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0301" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>95</mn><mo>%</mo></mrow></math> </ephtml> Hartung‐Knapp confidence intervals at the fixed truncation time <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0302" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>0.8</mn><mo>⋅</mo><msub><mi>τ</mi><mi>max</mi></msub></mrow></math> </ephtml> (logistic transformation, <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0303" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>30</mn></mrow></math> </ephtml> ). It is seen that the confidence intervals obtained from the meta‐analysis model (ignoring follow‐up time) did not reach the desired coverage probability in the scenario with long follow‐up. All other coverage probability estimates were close to the <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0304" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>95</mn><mo>%</mo></mrow></math> </ephtml> level. The results obtained from the models with untransformed and arcsine‐square‐root‐transformed <emph>C</emph>‐index estimates ( <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0305" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>30</mn></mrow></math> </ephtml> ) showed similar patterns (Figures S8 and S9, respectively), except that the meta‐analysis model performed generally worse than the other models when fitted to the untransformed estimates (regardless of the length of follow‐up). The estimated coverage probabilities obtained from the scenarios with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0306" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>10</mn></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0307" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>15</mn></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0308" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>50</mn></mrow></math> </ephtml> showed similar patterns as well, again suggesting that the meta‐analysis model is inferior to the meta‐regression models in the scenarios with long follow‐up (data not shown).</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01nov23/jrsm1655-fig-0004.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1655-fig-0004.jpg" title="4 Results of the simulation study (K=30). The plot shows the estimated coverage probabilities (%) that is, the proportion of simulation runs in which the 95% Hartung‐Knapp confidence intervals contained the true value of C0.8⋅τmax. Confidence limits (represented by the black lines) were computed as p̂±1.96⋅p̂⋅1−p̂/1000, where p̂ denotes the point estimate of the coverage probability. The red lines refer to the 95% confidence level. All C‐index estimates were transformed by a logistic transformation before model fitting. [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <p>Table 1 presents the areas enclosed by the true and the estimated <emph>C</emph>‐index curves. It is seen that the areas obtained from time‐independent random‐effects meta‐analysis tended to increase with increasing follow‐up time, whereas the respective areas obtained from the meta‐regression models tended to <emph>de</emph>crease with increasing follow‐up time. Random‐effects meta‐analysis was the overall best method in settings with short follow‐up. By contrast, linear meta‐regression of logit‐transformed <emph>C</emph>‐index estimates and fractional polynomial meta‐regression of logit‐transformed <emph>C</emph>‐index estimates tended to perform best in the scenarios with moderate and long follow‐up times, respectively. These results clearly suggest that the time‐constant functions obtained from random‐effects meta‐analysis are reasonable approximations to <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0314" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> in settings with a short follow‐up. By contrast, the benefits of modeling <emph>C</emph>‐index values by a regression function become apparent when follow‐up times are "long enough" to demonstrate possible time dependencies and nonlinear shapes of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0315" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> (for a discussion on how to assess the relative length of the follow‐up time, see Section 4). In most cases, the areas between the true and the estimated curves were smallest when <emph>C</emph>‐index estimates were transformed by the logistic transformation before model fitting. Similar results were obtained in the scenarios with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0316" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>10</mn></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0317" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>15</mn></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0318" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>50</mn></mrow></math> </ephtml> (Tables S5–S7, respectively), except that the areas obtained from time‐independent meta‐analysis were considerably less dependent on follow‐up time when <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0319" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>10</mn></mrow></math> </ephtml> . In addition, linear meta‐regression of logit‐transformed <emph>C</emph>‐index estimates tended to perform best for long follow‐up lengths in this scenario (Table S5).</p> <p>1 TABLE Results of the simulation study (K=30).</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left"><p><math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0321" xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><msub><mi>σ</mi><mi>a</mi></msub><mo>=</mo><mn>0</mn></mrow></math></p></th><th align="left"><p><math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0322" xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><msub><mi>σ</mi><mi>a</mi></msub><mo>=</mo><mn>0.01</mn></mrow></math></p></th><th align="left"><p><math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0323" xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><msub><mi>σ</mi><mi>a</mi></msub><mo>=</mo><mn>0.03</mn></mrow></math></p></th></tr><tr><th align="left">Short</th><th align="left">Moderate</th><th align="left">Long</th><th align="left">Short</th><th align="left">Moderate</th><th align="left">Long</th><th align="left">Short</th><th align="left">Moderate</th><th align="left">Long</th></tr></thead><tbody valign="top"><tr><td>MA (id)</td><td>8.9 (4.9)</td><td>10.0 (2.0)</td><td>12.0 (1.5)</td><td>9.1 (5.1)</td><td>10.1 (2.1)</td><td>13.4 (2.5)</td><td>10.2 (6.1)</td><td>10.9 (3.3)</td><td>13.4 (2.5)</td></tr><tr><td>MA (id, last 50%)</td><td>8.4 (4.0)</td><td>11.7 (3.2)</td><td>11.7 (1.6)</td><td>8.6 (4.4)</td><td>11.9 (3.6)</td><td>14.1 (4.5)</td><td>10.7 (6.4)</td><td>13.4 (5.7)</td><td>14.1 (4.5)</td></tr><tr><td>MA (id, last 30%)</td><td>9.4 (5.0)</td><td>13.5 (4.4)</td><td>12.5 (2.5)</td><td>9.8 (5.6)</td><td>13.9 (5.1)</td><td>15.6 (5.8)</td><td>12.6 (8.3)</td><td>15.8 (8.0)</td><td>15.6 (5.8)</td></tr><tr><td>MA (logit)</td><td>7.9 (3.6)</td><td>10.8 (2.5)</td><td>11.4 (1.1)</td><td>8.2 (3.9)</td><td>10.8 (2.7)</td><td>12.9 (2.2)</td><td>9.7 (5.5)</td><td>11.5 (3.9)</td><td>12.9 (2.2)</td></tr><tr><td>MA (logit, last 50%)</td><td>9.2 (4.6)</td><td>12.7 (3.6)</td><td>11.9 (1.8)</td><td>9.5 (4.9)</td><td>12.9 (4.0)</td><td>14.1 (4.5)</td><td>11.4 (6.9)</td><td>14.1 (6.2)</td><td>14.1 (4.5)</td></tr><tr><td>MA (logit, last 30%)</td><td>10.4 (5.6)</td><td>14.4 (4.8)</td><td>12.7 (2.6)</td><td>10.8 (6.1)</td><td>14.7 (5.5)</td><td>15.5 (5.8)</td><td>13.5 (8.7)</td><td>16.4 (8.3)</td><td>15.5 (5.8)</td></tr><tr><td>MA (asin)</td><td>8.1 (4.2)</td><td>10.2 (2.1)</td><td>11.6 (1.3)</td><td>8.3 (4.4)</td><td>10.3 (2.2)</td><td>13.1 (2.3)</td><td>9.6 (5.6)</td><td>10.9 (3.3)</td><td>13.1 (2.3)</td></tr><tr><td>MA (asin, last 50%)</td><td>8.6 (4.2)</td><td>12.2 (3.4)</td><td>11.8 (1.7)</td><td>8.9 (4.5)</td><td>12.4 (3.8)</td><td>14.1 (4.5)</td><td>10.9 (6.5)</td><td>13.6 (5.9)</td><td>14.1 (4.5)</td></tr><tr><td>MA (asin, last 30%)</td><td>9.8 (5.3)</td><td>13.9 (4.6)</td><td>12.6 (2.5)</td><td>10.2 (5.8)</td><td>14.3 (5.3)</td><td>15.5 (5.8)</td><td>12.9 (8.4)</td><td>16.0 (8.1)</td><td>15.5 (5.8)</td></tr><tr><td>linear (id)</td><td>13.4 (7.6)</td><td>10.0 (6.2)</td><td>7.0 (2.2)</td><td>13.3 (7.6)</td><td>10.2 (6.2)</td><td>9.7 (3.8)</td><td>13.5 (7.9)</td><td>11.5 (6.4)</td><td>9.7 (3.8)</td></tr><tr><td>linear (logit)</td><td>9.5 (5.7)</td><td>7.4 (4.3)</td><td>6.9 (1.6)</td><td>9.7 (5.9)</td><td>7.8 (4.4)</td><td>9.5 (3.6)</td><td>11.4 (6.8)</td><td>10.2 (5.4)</td><td>9.5 (3.6)</td></tr><tr><td>linear (asin)</td><td>11.2 (6.7)</td><td>8.4 (5.2)</td><td>6.8 (1.7)</td><td>11.3 (6.8)</td><td>8.7 (5.3)</td><td>9.6 (3.7)</td><td>12.1 (7.2)</td><td>10.6 (5.8)</td><td>9.6 (3.7)</td></tr><tr><td>RCS (id)</td><td>16.0 (6.9)</td><td>12.6 (5.7)</td><td>6.9 (3.0)</td><td>16.2 (6.8)</td><td>12.9 (5.7)</td><td>11.5 (4.4)</td><td>17.6 (7.2)</td><td>15.1 (6.0)</td><td>11.5 (4.4)</td></tr><tr><td>RCS (logit)</td><td>13.4 (5.8)</td><td>10.8 (4.6)</td><td>6.2 (2.5)</td><td>13.7 (5.9)</td><td>11.2 (4.6)</td><td>11.2 (4.3)</td><td>16.0 (6.6)</td><td>14.1 (5.5)</td><td>11.2 (4.3)</td></tr><tr><td>RCS (asin)</td><td>14.5 (6.4)</td><td>11.5 (5.1)</td><td>6.4 (2.7)</td><td>14.8 (6.4)</td><td>11.9 (5.1)</td><td>11.3 (4.3)</td><td>16.6 (6.8)</td><td>14.5 (5.7)</td><td>11.3 (4.3)</td></tr><tr><td>FP2 (id)</td><td>14.4 (6.9)</td><td>11.2 (5.6)</td><td>6.3 (2.7)</td><td>14.6 (6.9)</td><td>11.5 (5.6)</td><td>9.8 (4.1)</td><td>15.5 (7.3)</td><td>13.2 (6.0)</td><td>9.8 (4.1)</td></tr><tr><td>FP2 (logit)</td><td>11.4 (5.7)</td><td>9.3 (4.4)</td><td>5.7 (2.4)</td><td>11.7 (5.9)</td><td>9.7 (4.5)</td><td>9.7 (4.1)</td><td>13.6 (6.7)</td><td>12.2 (5.3)</td><td>9.7 (4.1)</td></tr><tr><td>FP2 (asin)</td><td>12.7 (6.3)</td><td>10.0 (5.0)</td><td>5.9 (2.5)</td><td>12.9 (6.4)</td><td>10.4 (5.0)</td><td>9.7 (4.1)</td><td>14.3 (7.0)</td><td>12.6 (5.6)</td><td>9.7 (4.1)</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note</emph>: The table summarizes the areas enclosed by the true and the estimated <emph>C</emph>‐index curves (mean [sd]), as obtained from the meta‐regression models described in Section 2. All areas were divided by the interval lengths <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0324" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mfenced open="(" close=")"><mrow><msub><mi>max</mi><mi>k</mi></msub><mfenced open="(" close=")"><msub><mi>τ</mi><mi>k</mi></msub></mfenced><mo>−</mo><msub><mi>min</mi><mi>k</mi></msub><mfenced open="(" close=")"><msub><mi>τ</mi><mi>k</mi></msub></mfenced></mrow></mfenced></mrow></math> </ephtml> and multiplied by 1000.</p> <p>In summary, our simulation study suggests that (i) spline and fractional polynomial meta‐regression should be preferred over the exponential decay approach to model nonlinearities in <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0325" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> , (ii) pooled estimates obtained from time‐independent meta‐analysis are reasonable approximations of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0326" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> as long as follow‐up times short, whereas it is beneficial to consider increasingly complex meta‐regression models with increasing values of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0327" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0328" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi></mrow></math> </ephtml> , and (iii) models with logit‐transformed <emph>C</emph>‐index estimates showed the overall best performance (compared to untransformed and arcsine‐square‐root‐transformed estimates).</p> <p>We further note that, compared to spline meta‐regression, fractional polynomial meta‐regression showed a slightly better overall performance in terms of the areas enclosed between the true and the fitted <emph>C</emph>‐index curves. Importantly, the performance of fractional polynomial meta‐regression could be improved further by optimizing the power values <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0329" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>p</mi><mn>1</mn></msub></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0330" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>p</mi><mn>2</mn></msub></mrow></math> </ephtml> (instead of considering the fixed values <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0331" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>−</mo><mn>0.5</mn></mrow></math> </ephtml> and <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0332" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>0.5</mn></mrow></math> </ephtml> , as done in this section). We will investigate this issue further in Section 4. For details on variable and power selection in fractional polynomial regression, see Royston and Sauerbrei.[<reflink idref="bib38" id="ref82">38</reflink>]</p> <hd id="AN0173455155-19">ILLUSTRATION</hd> <p>To illustrate the proposed methods, we analyzed data from the German Chronic Kidney Disease (GCKD) Study, which is an ongoing multi‐center cohort study that enrolled 5217 patients with chronic kidney disease (CKD). The aim of the study is to identify risk factors associated with CKD progression, cardiovascular events and death. For details on the inclusion/exclusion criteria and the design of the study, see Eckardt et al.[<reflink idref="bib24" id="ref83">24</reflink>] Baseline data collection took place between March 2010 and March 2012; it comprised measurements on clinical and lifestyle variables (e.g. coronary heart disease, smoking) and biomarker measurements obtained from blood and urine samples. Follow‐up data are collected annually. The laboratory measurements collected for the GCKD Study have been used previously for predictive modeling and score development.[<reflink idref="bib25" id="ref84">25</reflink>]</p> <p>An important characteristic of the GCKD Study is its wide geographical coverage. Altogether, there are nine study centers, each representing a specific German region with a distinct patient population. During the past years, it has become increasingly popular to account for such heterogeneity by synthesizing center‐specific estimates via meta‐analysis techniques.[[<reflink idref="bib44" id="ref85">44</reflink>], [<reflink idref="bib46" id="ref86">46</reflink>]] Here we followed this approach and used the GCKD data to evaluate a prognostic model in each of the nine centers, illustrating our proposed methodology by meta analyzing the respective center‐specific <emph>C</emph>‐index estimates ( <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0333" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>K</mi><mo>=</mo><mn>9</mn></mrow></math> </ephtml> ). Note that the availability of individual patient data allowed us to estimate <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0334" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><mi>τ</mi></mfenced></mrow></math> </ephtml> in each center at arbitrary time horizons.</p> <p>For model building and evaluation we considered the endpoint "time to cardiovascular death". Data were exported from the GCKD database after the 8th follow‐up examination (maximum follow‐up time 2933 days, median = 2554 days, first quartile = 2060 days, third quartile = 2591 days, cardiovascular death rate = 200/4455 = 4.5% after listwise deletion of patients with a missing value in at least one of the covariates). In the first step, we split the data randomly into three equally sized parts: The first part was used as <emph>training data</emph> for model building, the second part was used as <emph>analysis data</emph> for prediction and meta‐regression, and the third part was used as <emph>test data</emph> for evaluating the performance of the meta‐regression models. In the second step, we derived a prediction model for cardiovascular death by fitting a Cox regression model to the training data. The following (pre‐selected) baseline covariates were included in the model: C‐reactive protein (mg/L), cholesterol (mg/dL), calcium (mmol/L), phosphate (mmol/L), albumin (g/L), cystatin C (mg/L), age (years), sex (male/female), urine albumin‐to‐creatinine ratio (mg/g), hypertension (yes/no), previous coronary heart disease (yes/no), smoking (non‐smoker, former smoker, current smoker), and estimated glomerular filtration rate (mL/min/1.73 m<sups>2</sups>). Furthermore, we generated a random truncation time <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0335" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0336" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>k</mi><mo>=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mn>9</mn></mrow></math> </ephtml> , for each of the study centers, restricting <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0337" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> to be larger than 3.5 years in order to obtain sufficiently large event counts. In the third step, we used the coefficients of the Cox model to predict the values of <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0338" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>η</mi></mrow></math> </ephtml> in the analysis data. These values were subsequently used to estimate the restricted concordance probability in each study center at the center‐specific truncation times <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0339" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>τ</mi><mi>k</mi></msub></mrow></math> </ephtml> . The resulting estimates <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0340" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub></mrow></math> </ephtml> (obtained by application of Uno's <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0341" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi></mrow></math> </ephtml> ) are visualized in Figure 5a. In the fourth step, we meta‐analyzed the center‐specific <emph>C</emph>‐index estimates by applying the methods presented in Section 2.4 to the pairs of values <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0342" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mfenced open="(" close=")" separators=","><msub><mi>τ</mi><mn>1</mn></msub><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mn>1</mn></msub></mfenced><mo>,</mo><mi>...</mi><mo>,</mo><mfenced open="(" close=")" separators=","><msub><mi>τ</mi><mn>9</mn></msub><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mn>9</mn></msub></mfenced></mrow></math> </ephtml> . Based on the results of our simulation study, <emph>C</emph>‐index estimates were logit‐transformed before model fitting. In the fifth step, we generated 1000 bootstrap samples from the test data and re‐estimated the concordance probabilities <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0343" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced open="(" close=")"><msub><mi>τ</mi><mi>k</mi></msub></mfenced></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0344" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>k</mi><mo>=</mo><mn>1</mn><mo>,</mo><mi>...</mi><mo>,</mo><mn>9</mn></mrow></math> </ephtml> , as well as the fitted values <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0345" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msup><mi>g</mi><mrow><mo>−</mo><mn>1</mn></mrow></msup><mfenced open="(" close=")"><mrow><mi>f</mi><mfenced open="(" close=")" separators=";"><msub><mi>τ</mi><mi>k</mi></msub><mover accent="true"><mi>γ</mi><mo>̂</mo></mover></mfenced></mrow></mfenced></mrow></math> </ephtml> (obtained from meta‐regression) in each of the samples. Furthermore, we computed the weighted root mean squared error (RMSE, defined by <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0346" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msup><mfenced open="[" close="]"><mrow><msubsup><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><mi>K</mi></msubsup><msub><mi>n</mi><mi>k</mi></msub><mo>/</mo><mi>n</mi><mspace width="0.1em" /><msup><mfenced open="(" close=")"><mrow><msub><mover accent="true"><mi>C</mi><mo>̂</mo></mover><mi>k</mi></msub><mo>−</mo><msup><mi>g</mi><mrow><mo>−</mo><mn>1</mn></mrow></msup><mfenced open="(" close=")"><mrow><mi>f</mi><mfenced open="(" close=")" separators=";"><msub><mi>τ</mi><mi>k</mi></msub><mover accent="true"><mi>γ</mi><mo>̂</mo></mover></mfenced></mrow></mfenced></mrow></mfenced><mn>2</mn></msup></mrow></mfenced><mrow><mn>1</mn><mo>/</mo><mn>2</mn></mrow></msup></mrow></math> </ephtml> ), which was used to evaluate and compare the performance of the meta‐regression models.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01nov23/jrsm1655-fig-0005.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1655-fig-0005.jpg" title="5 Analysis of the German Chronic Kidney Disease (GCKD) Study data. (a) Nine center‐specific C‐index estimates at randomly generated truncation times τk, k=1,...,9. Estimates and 95% confidence intervals (represented by bars) were obtained by application of Uno's C to the GCKD analysis data. (b) The colored lines refer to the back‐transformed meta‐analysis and ‐regression curves obtained by fitting the models of Section 2.4 to the logit‐transformed C‐index estimates (FP2, fractional polynomial meta‐regression; linear, linear meta‐regression; MA, standard random effects meta‐analysis ignoring time dependency; RCS, restricted cubic spline meta‐regression). (c) Root mean squared error values obtained from the bootstrapped test data (1000 replications). [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <p>The fitted curves obtained from the analysis data are presented in Figure 5b. It is seen that the meta‐regression models (accounting for the length of follow‐up) resulted in very similar fits. Fractional polynomial meta‐regression seemed to perform best by visual inspection. The model summaries (given in Table 2) confirm these results, with the values of the estimated between‐study standard deviation <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0351" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>σ</mi><mo>̂</mo></mover><mi>a</mi></msub></mrow></math> </ephtml> ranging between 0.708 and 0.821 on the logit scale. Boxplots of the RMSE values (obtained from the bootstrapped test data) are shown in Figure 5c. Again, it can be seen that the models performed similarly, with the highest RMSE value observed for the meta‐analysis model and the lowest RMSE value for the fractional polynomial meta‐regression model.</p> <p>2 TABLE Analysis of the GCKD Study data.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left"><p><math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0352" xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><msub><mover accent="true"><mi>σ</mi><mo>̂</mo></mover><mi>a</mi></msub></mrow></math></p></th><th align="left"><p><math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0353" xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mi>Q</mi></mrow></math></p></th><th align="left">df</th><th align="left"><italic>p</italic> value</th></tr></thead><tbody valign="top"><tr><td>MA (logit)</td><td>0.708</td><td>22.3</td><td>8</td><td>0.0043</td></tr><tr><td>Linear (logit)</td><td>0.711</td><td>19.9</td><td>7</td><td>0.0059</td></tr><tr><td>RCS (logit)</td><td>0.821</td><td>18.9</td><td>6</td><td>0.0043</td></tr><tr><td>FP2 (logit)</td><td>0.793</td><td>18.1</td><td>6</td><td>0.0060</td></tr></tbody></table> </ephtml> </p> <p>2 <emph>Note</emph>: The table summarizes the fits of the meta‐analysis and ‐regression models, as obtained by applying the methods of Section 2.4 to the center‐specific <emph>C</emph>‐index values (estimated from the GCKD analysis data). The logistic transformation was applied to the estimated <emph>C</emph>‐index values before model fitting. The table presents the values of the estimated between‐study standard deviation <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0354" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover accent="true"><mi>σ</mi><mo>̂</mo></mover><mi>a</mi></msub></mrow></math> </ephtml> (on the logit scale), the test statistic for residual heterogeneity <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0355" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>Q</mi></mrow></math> </ephtml> (following a Chi‐squared distribution under the null hypothesis of homogeneous residuals), its degrees of freedom (df), and the corresponding <emph>p</emph> value.</p> <p>In the final step, we investigated whether we could improve the performance of fractional polynomial meta‐regression by optimizing the power values of the FP2 model. To this purpose, we repeated the bootstrap analysis of the GCKD test data, this time computing the RMSE values obtained from all possible combinations of the powers <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0356" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>p</mi><mn>1</mn></msub><mo>,</mo><msub><mi>p</mi><mn>2</mn></msub><mo>∈</mo><mfenced open="{" close="}"><mrow><mo>−</mo><mn>2</mn><mo>,</mo><mo>−</mo><mn>1</mn><mo>,</mo><mo>−</mo><mn>0.5,0,0.5,1,2,3</mn></mrow></mfenced></mrow></math> </ephtml> . The results of our analysis suggest that the RMSE values were not very sensitive to the choice of powers (Figure 6). In particular, the performance of our initial model from Section 2.4 ( <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0357" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>p</mi><mn>1</mn></msub><mo>=</mo><mo>−</mo><mn>0.5</mn></mrow></math> </ephtml> , <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0358" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>p</mi><mn>2</mn></msub><mo>=</mo><mn>0.5</mn></mrow></math> </ephtml> , Figure 5c) was close to the performance of the optimal model with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0359" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>p</mi><mn>1</mn></msub><mo>=</mo><msub><mi>p</mi><mn>2</mn></msub><mo>=</mo><mo>−</mo><mn>2</mn></mrow></math> </ephtml> . In simulations (based on the same design as in Section 3), we additionally investigated the use of 3rd degree fractional polynomials for meta‐regression of the <emph>C</emph>‐index. Our results (not shown) suggest that increasing the degree of the fractional polynomial had very little effect on the median <emph>C</emph>‐index estimates at <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0360" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>0.8</mn><mo>⋅</mo><msub><mi>τ</mi><mi>max</mi></msub></mrow></math> </ephtml> . Instead, this approach tended to increase both the variance of the estimates and the areas enclosed by the true and estimated <emph>C</emph>‐index curves.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/BDCT/01nov23/jrsm1655-fig-0006.jpg?ephost1=dGJyMMvl7ESepq84yOvsOLCmsE6epq5Srqa4SK6WxWXS" alt="jrsm1655-fig-0006.jpg" title="6 Analysis of the German Chronic Kidney Disease (GCKD) data. The boxplots show the root mean squared error (RMSE) values obtained from the bootstrapped test data (1000 replications) when evaluating all possible combinations of the power values p1,p2∈−2,−1,−0.5,0,0.5,1,2,3 of the FP2 models. The boxplots are ordered by median RMSE value. The dark orange boxplot corresponds to the powers of the FP2 model from Section 2.4 (p1=−0.5, p2=0.5). [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <hd id="AN0173455155-22">DISCUSSION</hd> <p>The development of prognostic models has become a predominant task in medical and epidemiological research. As noted by Riley et al.,[<reflink idref="bib5" id="ref87">5</reflink>] "prognostic factors have many potential uses, including aiding treatment and lifestyle decisions, improving individual risk prediction, providing novel targets for new treatment, and enhancing the design and analysis of randomized trials identify patients for trials". To this end, a large number of prognostic scores has been developed, requiring proper validation to become accepted for eventual use in clinical practice. The gold standard for validation is to analyze the performance of novel scores using large external cohorts; however, for a variety of reasons (including confidentiality and logistical issues), this is often not possible. It is therefore important to combine the results of smaller validation studies using meta‐analysis techniques.</p> <p>In this paper we proposed and evaluated a framework for meta‐analyzing the concordance index for time‐to‐event data, which has become an established measure for the discriminative ability of prognostic scores (see Steyerberg[<reflink idref="bib8" id="ref88">8</reflink>] for a comprehensive introduction to predictive modeling, also including other aspects of validation like calibration and clinical usefulness). We analyzed the inherent time‐dependency of <emph>C</emph>‐index estimates (noted previously by Longato et al.[<reflink idref="bib30" id="ref89">30</reflink>]) and proposed methods to account for this time‐dependency using meta‐regression models. In this respect, our paper connects to Debray et al.[<reflink idref="bib9" id="ref90">9</reflink>] who noted that "[...] researchers often refrain from undertaking a quantitative synthesis or meta‐analysis of the predictive performance of a specific model. Potential reasons for this pitfall are [...] or simply a lack of methodological guidance."</p> <p>A key result of this work is that meta‐regression models including the study‐specific truncation time as covariate perform systematically better than classical random‐effects meta‐analysis when follow‐up times of validation studies are long. Conversely, pooled estimates obtained from classical meta‐analysis are reasonable approximations of the restricted concordance probability as long as follow‐up times short. We acknowledge that, in practice, it might be challenging to determine whether a time horizon should be considered "short" or "long", in particular when observation times are affected by high dropout rates and/or the presence of competing events. Still, we recommend to carefully investigate this issue, especially since <emph>C</emph>‐index values often tend to decrease with <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0364" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi></mrow></math> </ephtml> , implying that studies with a short follow‐up time might suggest an overly optimistic discrimination accuracy. Generally, the rate of administrative censoring might be an indicator of whether follow‐up times might be considered "long" or "short". Furthermore, we recommend visual inspection of <emph>C</emph>‐index estimates in order to examine their dependency on <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0365" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>τ</mi></mrow></math> </ephtml> .</p> <p>Based on our numerical experiments, we recommend to transform <emph>C</emph>‐index values using a logistic transformation and to employ either restricted cubic splines or fractional polynomials to model the functional relationship between the truncation time and the concordance index. We further recommend to prefer fractional polynomials over splines in settings where the number of studies is "small" ( <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0366" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>5</mn><mo>≤</mo><mi>K</mi><mo>≤</mo><mn>10</mn></mrow></math> </ephtml> ), as they typically involve fewer degrees of freedom than restricted cubic splines. In case of convergence problems (which might become an issue when the number of studies is smaller than five), our framework readily allows for switching to a simpler model (e.g. a linear meta‐regression model). We also note that our proposed models could be extended by additional covariates reflecting different inclusion criteria in the analyzed studies. Along the same lines, our framework could be adapted to models with competing events.[<reflink idref="bib48" id="ref91">48</reflink>]</p> <p>Generally, by building on the framework of Debray et al.,[[<reflink idref="bib6" id="ref92">6</reflink>], [<reflink idref="bib9" id="ref93">9</reflink>]] our methodology is designed to synthesize <emph>C</emph>‐index estimates referring to the <emph>same</emph> score (e.g., the Framingham Risk Score). Technically, one could also imagine using our methods for a synthesis of <emph>C</emph>‐index estimates obtained from <emph>different</emph> scores. However, such analyses are rarely done in practice, which is likely due to the typically high heterogeneity between the included models (see e.g. Büttner et al.,[<reflink idref="bib20" id="ref94">20</reflink>] who considered a total of 18 prognostic models, but meta‐analyzed only those <emph>C</emph>‐index estimates referring to the same model). In view of these aspects, our perspective on the analysis of <emph>different</emph> scores is that performance estimates referring to different scores should better not be synthesized directly; instead, comparisons of different scores should be done by computing performance estimates from the same validation data (using the same truncation time for each score). As an alternative, one could borrow information from the other scores when analyzing one of them. This could be achieved by shrinkage estimation, e.g. in a Bayesian framework.[<reflink idref="bib49" id="ref95">49</reflink>] A key barrier to meta‐analyzing <emph>C</emph>‐index values is the huge variety of estimators that have been proposed during the past decades (such as Harrell's <ephtml> <math display="inline" overflow="scroll" altimg="urn:x-wiley:17592879:media:jrsm1655:jrsm1655-math-0367" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi></mrow></math> </ephtml> and Uno's <emph>C</emph>).[<reflink idref="bib15" id="ref96">15</reflink>] Since each of these estimators comes with a different set of assumptions and/or properties, it is challenging to synthesize validation studies relying on different kinds of estimators. Importantly, some of the estimators are known to be systematically biased, e.g. when they rely on a Cox model (whose assumptions might be violated) or when they show a censoring bias (such as Harrell's <emph>C</emph>). We argue that these systematic deviations should not be represented in a meta‐regression model by zero‐mean random effects. Instead, we suggest to develop methodological guidance on the definition and use of appropriate estimators for the evaluation of discriminatory power, aiming at a unified methodology that would become a standard in future validation studies. Work on such guidance is currently undertaken by the STRengthening Analytical Thinking for Observational Studies (STRATOS) initiative.[<reflink idref="bib50" id="ref97">50</reflink>]</p> <p>Meta‐regression of <emph>C</emph>‐index estimates is also compromised by the lack of proper reporting. In fact, when searching for a real‐world application to be presented in Section 4, we found that most published studies reporting <emph>C</emph>‐index estimates did <emph>not</emph> include any information on the respective time horizon. In some cases, we were able to approximate this time horizon by the length of the respective follow‐up time; however, in many cases the time horizon was not mentioned at all. Based on the findings presented in Section 3 of this paper, we suggest to always report the time horizons together with <emph>C</emph>‐index estimates in future validation studies. We further suggest to report and visualize the whole estimated <emph>C</emph>‐index curve whenever a meta‐regression has been performed. Ideally, reporting would also include time‐dependent sensitivities and specificities in addition to <emph>C</emph>‐index estimates, allowing for the application of bivariate meta‐analysis techniques.[<reflink idref="bib51" id="ref98">51</reflink>]</p> <p>We finally note that the concordance index is (by far) not the only prognostic measure to be affected by an inherent time dependency. Another important example are incidence rates, which by definition depend on the time frames under consideration. Clearly, the lengths of these time frames have to be considered when meta‐analyzing incidence rates (see Olaciregui‐Dague et al.[<reflink idref="bib52" id="ref99">52</reflink>] for a recent example). Further research is needed to evaluate possible adaptions of our methodology to these measures.</p> <hd id="AN0173455155-23">AUTHOR CONTRIBUTIONS</hd> <p> <bold>Matthias Schmid:</bold> Conceptualization; formal analysis; methodology; writing – original draft; writing – review and editing. <bold>Tim Friede:</bold> Methodology; writing – review and editing. <bold>Nadja Klein:</bold> Writing – review and editing. <bold>Leonie Weinhold:</bold> Conceptualization; formal analysis; methodology; software; visualization; writing – review and editing.</p> <hd id="AN0173455155-24">ACKNOWLEDGMENTS</hd> <p>We thank the GCKD Study Investigators for providing data of the GCKD Study for illustrative purposes. The GCKD study was supported by a grant from the KfH Foundation for Preventive Medicine (https://<ulink href="http://www.kfh-stiftung-praeventivmedizin.de">www.kfh-stiftung-praeventivmedizin.de</ulink>). Tim Friede is grateful for support by the Volkswagen Foundation (Az.: 98 948; "Bayesian and Nonparametric Statistics‐Teaming up two opposing theories for the benefit of prognostic studies in Covid‐19"). Open Access funding enabled and organized by Projekt DEAL.</p> <hd id="AN0173455155-25">DATA AVAILABILITY STATEMENT</hd> <p>The R scripts for the simulation study are available on GitHub at https://github.com/weinholdl/metac. The GCKD Study data contain information that could compromise the privacy of study participants. Data sharing restrictions imposed by national and transnational data sharing laws prohibit general sharing of these data. Upon submission of a proposal to the principal investigator of the GCKD Study and approval of this proposal by the Steering Committee of the GCKD Study, data collected for the study can be made available to other researchers.</p> <p>GRAPH: Appendix S1: Supplementary Information.</p> <ref id="AN0173455155-26"> <title> REFERENCES </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> Fire M, Guestrin C. Over‐optimization of academic publishing metrics: observing Goodhart's law in action. GigaScience. 2019 ; 8 (6): giz053.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref2" type="bt">2</bibl> <bibtext> Landhuis E. Scientific literature: information overload. Nature. 2016 ; 535 : 457 ‐ 458.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref4" type="bt">3</bibl> <bibtext> Altbach PG, de Wit H. Too much academic research is being published. Int Higher Educ. 2018 ; 96 : 2 ‐ 3.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref6" type="bt">4</bibl> <bibtext> Gough D, Davies P, Jamtvedt G, et al. Evidence synthesis international (ESI): position statement. Syst Rev. 2020 ; 9 : 155.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref7" type="bt">5</bibl> <bibtext> Riley RD, Moons KGM, Snell KIE, et al. A guide to systematic review and meta‐analysis of prognostic factor studies. BMJ. 2019 ; 364 : k4597285.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref8" type="bt">6</bibl> <bibtext> Debray TPA, Damen JAAG, Riley RD, et al. A framework for meta‐analysis of prediction model studies with binary and time‐to‐event outcomes. Stat Methods Med Res. 2019 ; 28 : 2768 ‐ 2786.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref9" type="bt">7</bibl> <bibtext> Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMC Med. 2015 ; 13 (1): 1.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref75" type="bt">8</bibl> <bibtext> Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. 2nd ed. Springer ; 2019.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref11" type="bt">9</bibl> <bibtext> Debray TPA, Damen JAAG, Snell KIE, et al. A guide to systematic review and meta‐analysis of prediction model performance. BMJ. 2017 ; 356 : i6460.</bibtext> </blist> <blist> <bibtext> Harrell FE, Lee KL, Califf RM, Pryor DB, Rosati RA. Regression modeling strategies for improved prognostic prediction. Stat Med. 1984 ; 3 : 143 ‐ 152.</bibtext> </blist> <blist> <bibtext> Heagerty PJ, Zheng Y. Survival model predictive accuracy and ROC curves. Biometrics. 2005 ; 61 : 92 ‐ 105.</bibtext> </blist> <blist> <bibtext> Gerds TA, Kattan MW, Schumacher M, Yu C. Estimating a time‐dependent concordance index for survival prediction models with covariate dependent censoring. Stat Med. 2013 ; 32 : 2173 ‐ 2184.</bibtext> </blist> <blist> <bibtext> Gönen M, Heller G. Concordance probability and discriminatory power in proportional hazards regression. Biometrika. 2005 ; 92 : 965 ‐ 970.</bibtext> </blist> <blist> <bibtext> Uno H, Cai T, Pencina MJ, D'Agostino RB, Wei LJ. On the C‐statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Stat Med. 2011 ; 30 : 1105 ‐ 1117.</bibtext> </blist> <blist> <bibtext> Schmid M, Potapov S. A comparison of estimators to evaluate the discriminatory power of time‐to‐event models. Stat Med. 2012 ; 31 : 2588 ‐ 2609.</bibtext> </blist> <blist> <bibtext> Snell KIE, Ensor J, Debray TPA, Moons KGM, Riley RD. Meta‐analysis of prediction model performance across multiple studies: which scale helps ensure between‐study normality for the C‐statistic and calibration measures? Stat Methods Med Res. 2018 ; 27 : 3505 ‐ 3522.</bibtext> </blist> <blist> <bibtext> van Doorn S, Debray TPA, Kaasenbrood F, et al. Predictive performance of the CHA2DS2‐VASc rule in atrial fibrillation: a systematic review and meta‐analysis. J Thromb Haemost. 2017 ; 15 : 1065 ‐ 1077.</bibtext> </blist> <blist> <bibtext> van den Boorn HG, Engelhardt EG, van Kleef J, et al. Prediction models for patients with esophageal or gastric cancer: a systematic review and meta‐analysis. PLoS One. 2018 ; 13 (2): e0192310.</bibtext> </blist> <blist> <bibtext> He Y, Ong Y, Li X, et al. Performance of prediction models on survival outcomes of colorectal cancer with surgical resection: a systematic review and meta‐analysis. Surg Oncol. 2019 ; 29 : 196 ‐ 202.</bibtext> </blist> <blist> <bibtext> Büttner S, Galjart B, Beumer BR, et al. Quality and performance of validated prognostic models for survival after resection of intrahepatic cholangiocarcinoma: a systematic review and meta‐analysis. HPB. 2021 ; 23 : 25 ‐ 36.</bibtext> </blist> <blist> <bibtext> Kothari G, Korte J, Lehrer EJ, et al. A systematic review and meta‐analysis of the prognostic value of radiomics based models in non‐small cell lung cancer treated with curative radiotherapy. Radiother Oncol. 2021 ; 155 : 188 ‐ 203.</bibtext> </blist> <blist> <bibtext> Pennells L, Kaptoge S, White IR, Thompson SG, Wood AM. Emerging risk factors collaboration. Assessing risk prediction models using individual participant data from multiple studies. Am J Epidemiol. 2014 ; 179 : 621 ‐ 632.</bibtext> </blist> <blist> <bibtext> Hattori S, Zhou XH. Summary concordance index for meta‐analysis of prognosis studies with a survival outcome. Stat Med. 2021 ; 40 : 5218 ‐ 5236.</bibtext> </blist> <blist> <bibtext> Eckardt KU, Bärthlein B, Baid‐Agrawal S, et al. The German chronic kidney disease (GCKD) study: design and methods. Nephrol Dial Transplant. 2012 ; 27 : 1454 ‐ 1460.</bibtext> </blist> <blist> <bibtext> Zacharias HU, Altenbuchinger M, Schultheiss UT, et al. A predictive model for progression of CKD to kidney failure based on routine laboratory tests. Am J Kidney Dis. 2022 ; 79 : 217 ‐ 230.</bibtext> </blist> <blist> <bibtext> Brentnall AR, Cuzick J. Use of the concordance index for predictors of censored survival data. Stat Methods Med Res. 2018 ; 27 : 2359 ‐ 2373.</bibtext> </blist> <blist> <bibtext> Gerds TA. pec: Prediction error curves for risk prediction models in survival analysis. R package version 2022.05.04. 2022. Accessed July 8, 2023. https://CRAN.R-project.org/package=pec</bibtext> </blist> <blist> <bibtext> Song X, Zhou XH. A semiparametric approach for the covariate specific ROC curve with survival outcome. Stat Sin. 2008 ; 18 : 947 ‐ 965.</bibtext> </blist> <blist> <bibtext> van Geloven N, He Y, Zwinderman AH, Putter H. Estimation of incident dynamic AUC in practice. Comput Stat Data Anal. 2021 ; 154 : 107095.</bibtext> </blist> <blist> <bibtext> Longato E, Vettoretti M, Camillo BD. A practical perspective on the concordance index for the evaluation and selection of prognostic time‐to‐event models. J Biomed Inform. 2020 ; 108 : 103496.</bibtext> </blist> <blist> <bibtext> Sinha BK, Hartung J, Knapp G. Statistical Meta‐Analysis with Applications. Wiley ; 2011.</bibtext> </blist> <blist> <bibtext> Waldron L, Haibe‐Kains B, Culhane AC, et al. Comparative meta‐analysis of prognostic gene signatures for late‐stage ovarian cancer. J Natl Cancer Inst. 2014 ; 106 : dju049.</bibtext> </blist> <blist> <bibtext> van Klaveren D, Steyerberg EW, Perel P, Vergouwe Y. Assessing discriminative ability of risk models in clustered data. BMC Med Res Methodol. 2014 ; 14 : 5.</bibtext> </blist> <blist> <bibtext> Schwarzer G, Chemaitelly H, Abu‐Raddad LJ, Rücker G. Seriously misleading results using inverse of freeman‐Tukey double arcsine transformation in meta‐analysis of single proportions. Res Synth Methods. 2019 ; 10 : 476 ‐ 483.</bibtext> </blist> <blist> <bibtext> Viechtbauer W. Conducting meta‐analyses in R with the metafor package. J Stat Softw. 2010 ; 36 (3): 1 ‐ 48.</bibtext> </blist> <blist> <bibtext> Harrell FE. rms: Regression modeling strategies. R package version 6.3‐0. 2022. Accessed July 8, 2023. https://CRAN.R-project.org/package=rms</bibtext> </blist> <blist> <bibtext> Harrell FE. Regression Modeling Strategies: with Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. 2nd ed. Springer ; 2015.</bibtext> </blist> <blist> <bibtext> Royston P, Sauerbrei W. Multivariable Model‐Building: A Pragmatic Approach to Regression Anaylsis Based on Fractional Polynomials for Modelling Continuous Variables. Wiley ; 2008.</bibtext> </blist> <blist> <bibtext> Royston P, Altman DG. Regression using fractional polynomials of continuous covariates: parsimonious parametric modelling. J R Stat Soc Ser C. 1994 ; 43 : 429 ‐ 453.</bibtext> </blist> <blist> <bibtext> Royston P. Model selection for univariable fractional polynomials. Stata J. 2017 ; 17 : 619 ‐ 629.</bibtext> </blist> <blist> <bibtext> Balduzzi S, Rücker G, Schwarzer G. How to perform a meta‐analysis with R: a practical tutorial. Evid Based Ment Health. 2019 ; 22 : 153 ‐ 160.</bibtext> </blist> <blist> <bibtext> Pinheiro J, Bates D, R Core Team. nlme: linear and nonlinear mixed effects models. R package version 3.1‐160. 2022. Accessed July 8, 2023. https://CRAN.R-project.org/package=nlme</bibtext> </blist> <blist> <bibtext> Röver C, Friede T. Double arcsine transform not appropriate for meta‐analysis. Res Synth Methods. 2022 ; 13 : 645 ‐ 648.</bibtext> </blist> <blist> <bibtext> Franceschini N, Shara N, Wang H, et al. The association of genetic variants of type 2 diabetes with kidney function. Kidney Int. 2012 ; 82 : 220 ‐ 225.</bibtext> </blist> <blist> <bibtext> Beekman M, Blanche H, Perola M, et al. Genome‐wide linkage analysis for human longevity: genetics of healthy ageing study. Aging Cell. 2013 ; 12 : 184 ‐ 193.</bibtext> </blist> <blist> <bibtext> Jögi NO, Kitaba N, Storaas T, et al. Ascaris exposure and its association with lung function, asthma, and DNA methylation in northern Europe. J Allergy Clin Immunol. 2022 ; 149 : 1960 ‐ 1969.</bibtext> </blist> <blist> <bibtext> Collatuzzo G, Visci G, Violante FS, et al. Determinants of anti‐S immune response at 6 months after COVID‐19 vaccination in a multicentric European cohort of healthcareworkers—ORCHESTRA project. Front Immunol. 2022 ; 13 : 986085.</bibtext> </blist> <blist> <bibtext> van Geloven N, Giardiello D, Bonneville EF, et al. Validation of prediction models in the presence of competing risks: a guide through modern methods. BMJ. 2022 ; 377 : e069249.</bibtext> </blist> <blist> <bibtext> Röver C, Friede T. Dynamically borrowing strength from another study through shrinkage estimation. Stat Methods Med Res. 2020 ; 29 : 293 ‐ 308.</bibtext> </blist> <blist> <bibtext> Sauerbrei W, Abrahamowicz M, Altman DG, le Cessie S, Carpenter J, STRATOS Initiative. Strengthening analytical thinking for observational studies: the STRATOS initiative. Stat Med. 2014 ; 33 : 5413 ‐ 5432.</bibtext> </blist> <blist> <bibtext> Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005 ; 58 : 982 ‐ 990.</bibtext> </blist> <blist> <bibtext> Olaciregui‐Dague K, Weinhold L, Hoppe C, Schmid M, Surges R. Anti‐seizure efficacy and retention rate of carbamazepine is highly variable in randomized controlled trials: a meta‐analysis. Epilepsia Open. 2022 ; 7 : 556 ‐ 569.</bibtext> </blist> </ref> <aug> <p>By Matthias Schmid; Tim Friede; Nadja Klein and Leonie Weinhold</p> <p>Reported by Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib10" firstref="ref13"></nolink> <nolink nlid="nl2" bibid="bib12" firstref="ref14"></nolink> <nolink nlid="nl3" bibid="bib11" firstref="ref17"></nolink> <nolink nlid="nl4" bibid="bib13" firstref="ref18"></nolink> <nolink nlid="nl5" bibid="bib15" firstref="ref19"></nolink> <nolink nlid="nl6" bibid="bib16" firstref="ref21"></nolink> <nolink nlid="nl7" bibid="bib17" firstref="ref22"></nolink> <nolink nlid="nl8" bibid="bib19" firstref="ref23"></nolink> <nolink nlid="nl9" bibid="bib20" firstref="ref24"></nolink> <nolink nlid="nl10" bibid="bib22" firstref="ref25"></nolink> <nolink nlid="nl11" bibid="bib23" firstref="ref26"></nolink> <nolink nlid="nl12" bibid="bib14" firstref="ref31"></nolink> <nolink nlid="nl13" bibid="bib24" firstref="ref32"></nolink> <nolink nlid="nl14" bibid="bib25" firstref="ref33"></nolink> <nolink nlid="nl15" bibid="bib26" firstref="ref43"></nolink> <nolink nlid="nl16" bibid="bib27" firstref="ref50"></nolink> <nolink nlid="nl17" bibid="bib28" firstref="ref54"></nolink> <nolink nlid="nl18" bibid="bib29" firstref="ref55"></nolink> <nolink nlid="nl19" bibid="bib30" firstref="ref56"></nolink> <nolink nlid="nl20" bibid="bib31" firstref="ref58"></nolink> <nolink nlid="nl21" bibid="bib32" firstref="ref61"></nolink> <nolink nlid="nl22" bibid="bib33" firstref="ref63"></nolink> <nolink nlid="nl23" bibid="bib34" firstref="ref64"></nolink> <nolink nlid="nl24" bibid="bib35" firstref="ref67"></nolink> <nolink nlid="nl25" bibid="bib37" firstref="ref68"></nolink> <nolink nlid="nl26" bibid="bib38" firstref="ref69"></nolink> <nolink nlid="nl27" bibid="bib39" firstref="ref71"></nolink> <nolink nlid="nl28" bibid="bib40" firstref="ref73"></nolink> <nolink nlid="nl29" bibid="bib41" firstref="ref79"></nolink> <nolink nlid="nl30" bibid="bib42" firstref="ref80"></nolink> <nolink nlid="nl31" bibid="bib43" firstref="ref81"></nolink> <nolink nlid="nl32" bibid="bib44" firstref="ref85"></nolink> <nolink nlid="nl33" bibid="bib46" firstref="ref86"></nolink> <nolink nlid="nl34" bibid="bib48" firstref="ref91"></nolink> <nolink nlid="nl35" bibid="bib49" firstref="ref95"></nolink> <nolink nlid="nl36" bibid="bib50" firstref="ref97"></nolink> <nolink nlid="nl37" bibid="bib51" firstref="ref98"></nolink> <nolink nlid="nl38" bibid="bib52" firstref="ref99"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1399283
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Accounting for Time Dependency in Meta-Analyses of Concordance Probability Estimates
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Schmid%2C+Matthias%22">Schmid, Matthias</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-0788-0317">0000-0002-0788-0317</externalLink>)<br /><searchLink fieldCode="AR" term="%22Friede%2C+Tim%22">Friede, Tim</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5347-7441">0000-0001-5347-7441</externalLink>)<br /><searchLink fieldCode="AR" term="%22Klein%2C+Nadja%22">Klein, Nadja</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-5072-5347">0000-0002-5072-5347</externalLink>)<br /><searchLink fieldCode="AR" term="%22Weinhold%2C+Leonie%22">Weinhold, Leonie</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Research+Synthesis+Methods%22"><i>Research Synthesis Methods</i></searchLink>. 2023 14(6):807-823.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 17
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Probability%22">Probability</searchLink><br /><searchLink fieldCode="DE" term="%22Meta+Analysis%22">Meta Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Time%22">Time</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Bias%22">Bias</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1002/jrsm.1655
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1759-2879<br />1759-2887
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Recent years have seen the development of many novel scoring tools for disease prognosis and prediction. To become accepted for use in clinical applications, these tools have to be validated on external data. In practice, validation is often hampered by logistical issues, resulting in multiple small-sized validation studies. It is therefore necessary to synthesize the results of these studies using techniques for meta-analysis. Here we consider strategies for meta analyzing the concordance probability for time-to-event data ("C-index"), which has become a popular tool to evaluate the discriminatory power of prediction models with a right-censored outcome. We show that standard meta-analysis of the C-index may lead to biased results, as the magnitude of the concordance probability depends on the length of the time interval used for evaluation (defined e.g., by the follow-up time, which might differ considerably between studies). To address this issue, we propose a set of methods for random-effects meta-regression that incorporate time directly as covariate in the model equation. In addition to analyzing nonlinear time trends via fractional polynomial, spline, and exponential decay models, we provide recommendations on suitable transformations of the C-index before meta-regression. Our results suggest that the C-index is best meta-analyzed using fractional polynomial meta-regression with logit-transformed C-index values. Classical random-effects meta-analysis (not considering time as covariate) is demonstrated to be a suitable alternative when follow-up times are small. Our findings have implications for the reporting of C-index values in future studies, which should include information on the length of the time interval underlying the calculations.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Note
  Label: Notes
  Group: Note
  Data: https://github.com/weinholdl/metac
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1399283
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1399283
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1002/jrsm.1655
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 17
        StartPage: 807
    Subjects:
      – SubjectFull: Probability
        Type: general
      – SubjectFull: Meta Analysis
        Type: general
      – SubjectFull: Time
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Bias
        Type: general
    Titles:
      – TitleFull: Accounting for Time Dependency in Meta-Analyses of Concordance Probability Estimates
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Schmid, Matthias
      – PersonEntity:
          Name:
            NameFull: Friede, Tim
      – PersonEntity:
          Name:
            NameFull: Klein, Nadja
      – PersonEntity:
          Name:
            NameFull: Weinhold, Leonie
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2023
          Identifiers:
            – Type: issn-print
              Value: 1759-2879
            – Type: issn-electronic
              Value: 1759-2887
          Numbering:
            – Type: volume
              Value: 14
            – Type: issue
              Value: 6
          Titles:
            – TitleFull: Research Synthesis Methods
              Type: main
ResultId 1