How to adjust a meta-analysis benchmark without patient-level data

Let’s say you need to estimate 12-month overall survival for patients receiving a hypothetical medicine called Veltruximab.

There are plenty of published studies of Veltruximab in the same type of advanced cancer. Each reports the percentage of patients still alive after 12 months. You perform a meta-analysis and combine their results to produce one overall estimate.

That estimate becomes your 12-month overall survival benchmark for Veltruximab. It is the reference point against which another study, or perhaps a new treatment, will be judged. You plan to use it in an HTA submission, discussions with payers and briefings with clinicians.

But there is a problem.

Even if all the studies address the same broad question, they may recruit very different patients. One study may include mostly people receiving their first treatment. Another may include people whose cancer has already progressed through several treatments. The average age, disease burden, ability to manage everyday activities and sites of disease may also differ.

Combining the survival results without considering those differences can be like lumping together apples and oranges. It can give you a benchmark for the wrong group of patients.

When individual patient data are unavailable, study-level covariates can help.

Matching PICOS can still leave you comparing apples with oranges

Studies are usually selected for a meta-analysis using PICOS. This stands for population, intervention, comparator, outcomes and study design.

PICOS helps ensure that the studies address a similar question. It does not make their patients identical.

Imagine that every study includes adults with the same cancer, evaluates Veltruximab and reports 12-month overall survival. They may still differ in:

• The proportion receiving first-line treatment

• The average age

• The proportion with disease outside the main affected organ

• The amount of disease present at the start

• The proportion who need help with everyday activities, often described as performance status

• Whether the study is a clinical trial or an observational study

• When the survival clock starts, which is often called time zero

These differences can affect survival independently of Veltruximab.

We explored this wider problem in our insight on heterogeneity in indirect comparisons. Studies can appear to match while still answering meaningfully different questions.

A simple average may describe the studies but not your target population

Suppose 12-month survival for Veltruximab is 40% in one study and 70% in another.

That difference may reflect chance, differences in the patients recruited, differences in study methods or other care received alongside Veltruximab. A study with younger patients receiving first-line treatment would be expected to have better survival than a study with older patients receiving later-line treatment.

A meta-analysis combines the results across studies. Larger and more precise studies usually contribute more to the final estimate.

The combined result is not necessarily wrong. It describes the mix of studies included in the analysis. But that mix may not resemble the population you need to benchmark.

An estimate of overall survival across these studies, without considering the differences between them, can therefore give a biased estimate for your target population.[1,2]

Study-level covariates can make the benchmark more relevant

A covariate is a characteristic that may be related to the outcome.

When patient-level data are available, we can use each person’s characteristics. We might know their age, treatment line and disease burden, as well as how long they survived.

Published studies rarely provide that level of detail. They may still report summaries such as:

• Mean age: 62 years

• First-line treatment: 70%

• Disease outside the main affected organ: 25%

These are study-level covariates. Each value describes a whole study rather than an individual patient.

Meta-regression examines whether study outcomes tend to change as these reported characteristics change. In plain English, it asks questions such as:

Do studies with more first-line patients tend to report higher 12-month survival for Veltruximab?

The fitted model can then estimate survival for a chosen combination of study characteristics.[2,3]

This does not recreate the missing patient-level data. It uses the information that is available to make the benchmark more relevant to the population of interest.

Decide who the benchmark should represent before running the model

Every adjusted benchmark needs a target.

You could adjust to the average characteristics across the included Veltruximab studies. If the studies contain an average of 60% first-line patients, the model can estimate survival for a study with 60% first-line patients.

That answers a valid question:

What survival does the model predict for the average Veltruximab study in this evidence base?

It may not answer the question you actually care about.

If your target population contains 75% first-line patients, you may instead want the model to predict survival at 75%. The same applies to age, disease burden and every other covariate in the model.[3,4]

The studies do not need to match the target value exactly. A study with 40% first-line patients provides information from below the target. A study with 80% provides information from above it. Meta-regression uses the pattern across studies to predict the outcome at the chosen target.

The target must still be supported by the evidence. Predicting at 60% is more credible when the studies range from 30% to 90% than when every study lies between 5% and 20%.

The intercept is the prediction at the values chosen for centring

Meta-regression software produces a number called the intercept.

Its meaning depends on where the covariates have been centred. Centring simply means choosing a value to act as zero in the model.

Suppose the target is 60% first-line patients. The model expresses each study relative to that target:

• A study with 40% is 20 percentage points below it

• A study with 60% is at the target

• A study with 80% is 20 percentage points above it

On the scale used in the model, the intercept is the predicted result at the target value. With several covariates, it represents the prediction when all of them equal their chosen target values. The result may need to be converted back into a survival percentage before it is reported.[3,4]

If the covariates are centred on their average values across the literature, the intercept represents the average study profile. If they are centred on your target population, it represents that target profile.

It is therefore better to describe the result as a model-adjusted benchmark for a specified target profile, rather than simply calling it the adjusted pooled estimate.

Build the benchmark in seven steps

Step 1. Define the outcome and target population

Start with the decision you need the benchmark to support.

For Veltruximab, the outcome is the percentage of patients alive at 12 months. Define when those 12 months begin. Also define the patients, treatment setting and line of therapy that the benchmark should represent.

Do not start by asking which covariates are available. Start by defining the question.

Step 2. Select studies that answer a sufficiently similar question

Set the eligibility criteria before selecting the studies. Check that they evaluate Veltruximab in sufficiently similar populations and settings, measure survival in the same way and provide enough follow-up.

Pay particular attention to time zero. Survival measured from diagnosis is not directly comparable with survival measured from the start of second-line treatment.

Study-level adjustment cannot repair fundamentally different questions.

Step 3. Choose a small number of important covariates

Select characteristics that clinical knowledge and existing evidence suggest are related to the outcome.

For an overall survival benchmark, these are prognostic factors. A prognostic factor is something associated with a patient’s likely outcome, regardless of which treatment they receive.

Choose the covariates before looking for the model that gives the most favourable result. Testing large numbers of characteristics and keeping only those that appear important can produce findings that occurred by chance.[1,2]

Step 4. Extract the outcome and covariates from every study

For each Veltruximab study, record:

• The 12-month survival estimate

• The uncertainty around that estimate

• Each chosen study-level covariate

• How each covariate was defined

• Relevant study-design characteristics

• Missing information

Definitions matter. The percentage with liver involvement is not comparable if one study means any liver disease and another means disease affecting more than half of the liver.

Step 5. Check what the evidence can support

Look at the range of each covariate across the studies. Check whether the target population falls within those ranges.

Also look at how the covariates relate to each other. Studies with more first-line patients may also contain younger patients. If two characteristics move together, a small evidence base may not be able to separate their relationships with survival.

Missing data matter too. A covariate reported by only a few studies may be clinically important but impossible to use reliably.

Step 6. Fit a random-effects meta-regression

A random-effects model allows the true outcome to vary across studies, even after the measured covariates have been included. This is important because the published characteristics will rarely explain every difference.[1,2,8]

The analysis should give more influence to more precise studies. It should also keep the remaining variation between studies visible rather than assuming that the covariates have explained it all.

This is the point at which statistical expertise is needed. Survival percentages often need to be analysed on a transformed scale and then converted back into an understandable percentage.

Step 7. Predict survival for the target profile

Use the fitted model to predict 12-month survival for Veltruximab at the characteristics of the target population.

Report the target values alongside the result. An adjusted benchmark without those values is incomplete.

Also report the uncertainty around the estimate and the variation that remains between studies. A prediction interval can show how widely the result of a new, similar study might vary.[2,4,8]

Compare the adjusted benchmark with the unadjusted result. If they differ, explain which population differences appear to drive that change. Do not imply that adjustment has uncovered one certain truth.

A trial-level eligibility rule shifted the survival benchmark by up to 15 percentage points

Korn and colleagues analysed 2,100 participants from 42 phase II trials of metastatic melanoma, covering 70 trial arms. They wanted to create historical benchmarks for one-year overall survival and six-month progression-free survival in future trials.[9]

The overall one-year survival rate across the studies was 25.5%. But the trials differed in their eligibility rules. The researchers examined trial-level characteristics including whether a trial excluded people with brain, liver or other visceral metastases, whether it excluded people who had received previous treatment, and the year in which recruitment ended.[9]

Whether a trial excluded people with brain metastases remained an important predictor of overall survival after the other factors were considered. The researchers included this trial-level covariate in a logistic regression for one-year survival, alongside the patients’ performance status, sex and visceral disease.[9]

The result was not one universally adjusted benchmark. Predicted one-year survival ranged from 5.5% to 63.8% across the profiles reported by the authors. For patients with the same performance status, sex and visceral disease, the predicted survival rate was between 4.3 and 15.0 percentage points higher when the trial excluded people with brain metastases.[9]

After the combined adjustment, the remaining variation in one-year survival between trial arms was essentially eliminated. It was not eliminated for six-month progression-free survival, so the authors considered that benchmark more difficult to use reliably.[9]

This was not a pure study-level meta-regression because the model also used patient-level characteristics. But it shows the point clearly. A trial-level design choice materially changed the survival benchmark, even when the patient profile was otherwise held constant.

The number of studies limits how much you can adjust

In meta-regression, the unit of analysis is the study, not the patient.

Ten thousand patients spread across eight studies still provide only eight study-level observations for the regression.

Cochrane advises that meta-regression should generally not be considered with fewer than 10 studies. A commonly used rule is at least 10 studies for each characteristic examined, but even that may be too few when covariate values are similar, information is missing or several covariates are related to each other.[1]

The practical answer is not to add every available characteristic. Use the smallest set needed to address the main differences in prognosis.

Study-level adjustment is not patient-level adjustment

This is the central limitation.

Meta-regression examines relationships between studies. It does not prove that the same relationship exists between patients within those studies.[1,2,5]

Suppose Veltruximab studies with more disease outside the main affected organ report poorer survival. Those studies may also include more later-line patients, use different follow-up methods or come from different healthcare systems. The disease location may partly capture those other differences.

This problem is called ecological bias. The study-level relationship can be weaker, stronger or even run in the opposite direction to the patient-level relationship.[5]

Study-level meta-regression can also be much less able than a patient-level analysis to detect a genuine relationship. It is a practical option when patient-level data are unavailable, but it is not an equivalent replacement.[6]

The published covariates also contain uncertainty. A percentage reported by a study is an estimate from its own sample, not a perfectly measured value. Differences in definitions and missing data add further error, which can weaken or distort the relationship estimated by the model.[7]

The result should therefore be described as adjusted for reported study-level characteristics. It should never be presented as though individual patients had been matched or adjusted directly.

Adjustment cannot rescue studies that are not comparable

The model cannot correct for a characteristic that was not reported. Nor can it fix poor study quality, selective reporting or incompatible outcome definitions.

Study-level covariates can make a benchmark more relevant. They do not make every collection of studies comparable.

The strongest analysis is transparent about:

• Why each study was included

• Why each covariate was chosen

• Which information was missing

• How close the target profile was to the observed studies

• How much variation remained after adjustment

• Which assumptions could materially change the result

That is the difference between a model that produces a number and an evidence benchmark that can support a decision.

Take home

• Studies can meet the same PICOS criteria for a meta-analysis and still recruit very different patients.

• A combined survival estimate may describe the published studies but not the population that matters to your decision.

• Study-level covariates can produce a more relevant benchmark when patient-level data are unavailable.

• Define the target profile before running the model. The benchmark is conditional on those chosen values.

• Keep the covariate set small and clinically justified. The number of studies limits what the model can support.

• Report the uncertainty and remaining variation. Study-level adjustment is not patient-level adjustment.

References

1. Deeks JJ, Higgins JPT, Altman DG, McKenzie JE, Veroniki AA. Chapter 10: Analysing data and undertaking meta-analyses. In: Higgins JPT, Thomas J, Chandler J, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.5. Cochrane; 2024. https://training.cochrane.org/handbook/current/chapter-10

2. Thompson SG, Higgins JPT. How should meta-regression analyses be undertaken and interpreted? Statistics in Medicine. 2002;21(11):1559-1573. https://doi.org/10.1002/sim.1187

3. Pagoni P, Higgins JPT, Lawlor DA, et al. Meta-regression of genome-wide association studies to estimate age-varying genetic effects. European Journal of Epidemiology. 2024;39:257-270. https://doi.org/10.1007/s10654-023-01086-1

4. Viechtbauer W. Conducting meta-analyses in R with the metafor package. Journal of Statistical Software. 2010;36(3):1-48. https://doi.org/10.18637/jss.v036.i03

5. Greenland S, Morgenstern H. Ecological bias, confounding, and effect modification. International Journal of Epidemiology. 1989;18(1):269-274. https://doi.org/10.1093/ije/18.1.269

6. Lambert PC, Sutton AJ, Abrams KR, Jones DR. A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. Journal of Clinical Epidemiology. 2002;55(1):86-94. https://doi.org/10.1016/S0895-4356(01)00414-0

7. Tran PT, Guolo A. Modeling the role of baseline risk and additional study-level covariates in meta-analysis of treatment effects. Statistics in Medicine. 2025;44:e70278. https://doi.org/10.1002/sim.70278

8. Berkey CS, Hoaglin DC, Mosteller F, Colditz GA. A random-effects regression model for meta-analysis. Statistics in Medicine. 1995;14(4):395-411. https://doi.org/10.1002/sim.4780140406

9. Korn EL, Liu PY, Lee SJ, et al. Meta-analysis of phase II cooperative group trials in metastatic stage IV melanoma to determine progression-free and overall survival benchmarks for future phase II trials. Journal of Clinical Oncology. 2008;26(4):527-534. https://doi.org/10.1200/JCO.2007.12.7837