How to judge the certainty of evidence: a plain-English guide to GRADE

NHS England reviewed siltuximab for adults with idiopathic multicentric Castleman disease.

Within the same randomised trial, the evidence for time to next treatment was rated high certainty. The evidence for overall response was rated moderate certainty because the result was imprecise.[1]

Same medicine. Same trial. Different certainty.

How can that be?

Because certainty does not belong to a medicine, a trial or a systematic review as a whole. It belongs to the evidence for a specific outcome.

GRADE provides a structured way to judge that certainty. It helps reviewers explain how confident they are in an estimated treatment effect and, crucially, why.[2-4]

An effect estimate is only half the answer

A meta-analysis combines results from several studies to estimate the overall effect of a medicine. It might estimate whether the medicine improves survival, reduces symptoms or increases serious adverse events.

But the estimate alone does not tell a committee how much to trust it.

A result can look precise but come from studies at high risk of bias. Several studies can produce similar results while measuring an outcome that does not answer the decision question. A non-significant result can reflect too little evidence rather than no treatment effect.[2,5]

GRADE stands for Grading of Recommendations Assessment, Development and Evaluation. It asks how certain we are that the true effect is close enough to the estimated effect for the decision being made.[2-4]

The answer is reported as high, moderate, low or very low certainty.

GRADE judges each outcome separately

NICE reviewed evidence comparing ceftazidime-avibactam with imipenem-cilastatin for acute pyelonephritis. Within the same randomised trial, the evidence for all adverse effects was rated moderate certainty. The evidence for serious adverse events was rated low certainty.[6]

There were six serious adverse events among 68 patients receiving ceftazidime-avibactam and two among 67 receiving imipenem-cilastatin. The estimated relative risk was 2.96, but its 95% confidence interval ranged from 0.62 to 14.13.[6]

This did not establish that ceftazidime-avibactam caused more serious adverse events. Nor did it establish that the treatments were equally safe. It showed that the study could not reliably determine the difference.

The medicines and trial were the same. The outcomes and the amount of information available were different. That is why GRADE assesses the evidence for each outcome separately.[2,4,6]

This is why it is misleading to say that a trial, medicine or systematic review is simply “high quality according to GRADE”. Risk of bias can be judged within individual studies. GRADE brings that judgement together with four other problems to assess certainty in the evidence for each outcome.[2]

GRADE starts with study design, then tests the evidence

Here is how GRADE is implemented in practice.

Step 1. Prespecify the question and how certainty will be judged

Before reviewing the results, define the population, medicine, comparator, outcomes and time points. Decide which outcomes are critical and what size of difference would matter clinically.[2,5,7]

Also prespecify how each GRADE domain will be assessed. This includes the risk-of-bias tool, how inconsistency will be judged and explored, which differences may create indirectness, the decision thresholds used for imprecision and how publication bias will be investigated.[2,7]

Some criteria can be numerical. Others require structured judgement. Planning the approach in advance reduces the risk that the rules are changed to fit the results.

Step 2. Synthesize the evidence for each outcome

Bring together the relevant studies for each outcome. When the studies are sufficiently similar, combine their results in a meta-analysis. When pooling would be misleading, synthesize the findings narratively. GRADE applies in either case.[2]

The unit being graded is the body of evidence for that outcome, not one paper in isolation.

Step 3. Set the starting certainty

Evidence from randomised controlled trials usually starts at high certainty. Evidence from non-randomised studies usually starts at low certainty.[2,3]

This is a starting point, not the final judgement. Randomised evidence can be downgraded substantially. Non-randomised evidence can sometimes be upgraded.

There is also a technical exception. When non-randomised studies are assessed using ROBINS-I, a detailed tool for judging bias, reviewers may start at high certainty and then usually downgrade for confounding and patient selection.[2]

Step 4. Check five reasons for lower certainty

Reviewers assess five domains. Each one asks a different question about whether the result can be trusted.[2]

Certainty problem	What reviewers ask	What it may reveal for the evidence plan
Risk of bias	Could problems in how the studies were designed or conducted have distorted the result?	Randomisation, blinding, follow-up, outcome measurement or adjustment for confounding may need strengthening.
Inconsistency	Do the study results differ without a convincing explanation?	Populations, doses, settings or factors that change the treatment effect may differ between studies.
Indirectness	Does the evidence match the actual population, medicine, comparator and outcome question?	The programme may need the relevant comparator, population or an outcome that matters directly to patients.
Imprecision	Is the estimate too uncertain to rule out meaningfully different conclusions?	More patients, events or follow-up may be needed. Meaningful decision thresholds should be defined in advance.
Publication bias	Could studies or outcomes be missing because of their results?	Studies should be registered, prespecified outcomes reported and searches extended beyond journal publications.

Step 5. Downgrade when a problem is serious

A serious limitation usually lowers certainty by one level. A very serious limitation usually lowers it by two.[2]

For example, randomised evidence begins at high certainty. If the estimate is seriously imprecise, it falls to moderate. If the imprecision is very serious, it may fall from high to low.

But GRADE is not a points system. Reviewers must decide how serious each problem is, consider whether two domains are capturing the same concern and avoid counting it twice.[2,4]

Step 6. Consider whether non-randomised evidence should be upgraded

Non-randomised evidence may be upgraded when there is a very large treatment effect, a clear dose-response relationship, or when remaining confounding would be expected to reduce the observed effect rather than create it.[2,3]

Upgrading should be exceptional and justified as carefully as downgrading.

Step 7. Agree and report the final certainty rating

The final rating has a plain meaning:[2]

• High certainty: reviewers are very confident that the true effect is close to the estimate.
• Moderate certainty: the true effect is probably close, but it could be meaningfully different.
• Low certainty: confidence is limited and the true effect may be meaningfully different.
• Very low certainty: there is very little confidence in the estimate.

Cochrane recommends that at least two people make the judgements independently and resolve differences through discussion.[2]

Every downgrade, upgrade and close decision should be justified. The audit trail should show the starting certainty, the judgement in every domain, the reason for that judgement and the final rating. A Summary of Findings table can then present the effect estimate, number of participants and studies, certainty rating and reasons together.[2]

As the GRADE authors put it:

“The great merit of GRADE is not that it necessarily ensures reproducible judgments … but that it achieves explicit and transparent judgment.”[4]

GRADE does not remove judgement. It makes judgement visible and accountable.

Plan how every certainty domain will be judged

Prespecification applies across GRADE, but not every domain has a sensible numerical threshold.

For risk of bias, specify the assessment tool and how concerns within individual studies will inform the judgement across the body of evidence. The contribution of each study matters. A high-risk study that contributes little information should not necessarily carry the same weight as one that drives the result.[2,7]

For inconsistency, specify how differences between study results will be assessed and which patient, treatment or study characteristics could plausibly explain them. Statistical measures such as I-squared can help, but a single cutoff should not replace consideration of the direction and size of the effects or their confidence intervals.[2]

For indirectness, use the prespecified PICO as the reference point. PICO stands for population, intervention, comparator and outcomes. Reviewers must judge whether differences between the available evidence and the decision question are important enough to reduce confidence.[2]

For imprecision, define what would count as an important benefit or harm. The key question is whether the confidence interval includes effects that would lead to meaningfully different decisions.[2,5]

Suppose a committee would consider a two-month improvement in survival clinically important. A confidence interval compatible with no benefit and a four-month benefit crosses that decision threshold. The result may therefore be too imprecise to support one clear conclusion, even if its average estimate favours the medicine.

For publication bias, specify how missing studies or outcomes will be investigated. This might include searches beyond journal publications, comparison with study registrations and assessment of patterns such as small positive studies.[2]

These are planned decision rules, not a formula. They make the approach more predictable without pretending that every judgement can be reduced to a number. Every final decision still needs a clear reason and a complete audit trail.[2,4,7]

GRADE can turn uncertainty into an evidence plan

The five domains are not only labels applied at the end of a systematic review. They identify why evidence may fail to support a decision.

Risk of bias points towards changes in study design and conduct. Inconsistency may reveal important differences between studies that need investigating. Indirectness can expose a mismatch in the population, comparator or outcome. Imprecision shows where more patients, events or follow-up are needed. Publication bias reinforces the need to register studies and report every prespecified outcome.[2]

The answer is not to collect everything.

It is to identify the outcomes capable of changing the decision, then generate the minimum credible evidence needed to support them.

Certainty informs a decision but does not make it

High-certainty evidence does not mean a medicine should be recommended. It could show with high certainty that the medicine offers little benefit or causes important harm.

Low-certainty evidence does not mean the medicine is ineffective. It means the true effect remains uncertain.[3,5]

Committees still need to consider the size of the benefit, harms, patient priorities, costs, available alternatives, feasibility and equality. Certainty is one part of the decision, not the decision itself.[3,5,7-8]

HTA bodies use GRADE in different ways

GRADE is directly relevant to health technology assessment, but its use is not identical everywhere.

NICE uses GRADE when developing clinical guidelines. NHS England has used a modified GRADE approach in specialised commissioning evidence reviews, including the siltuximab review.[1,7,8]

Zorginstituut Nederland includes GRADE within its assessment of whether care meets the established science and practice standard. The Danish Medicines Council uses GRADE elements for direct evidence and a related framework called CINeMA for network meta-analysis.[9,10]

IQWiG in Germany follows related evidence principles but uses its own categories when judging certainty about the added benefit of medicines.[11]

The lesson is not that one GRADE template will satisfy every market. It is that decision-makers expect uncertainty to be assessed transparently, outcome by outcome.

The final judgement must remain expert and traceable

Technology can accelerate searching, extraction and evidence mapping. It should not silently decide whether evidence is high, moderate, low or very low certainty.

Evidax TRACE supports evidence synthesis with expert oversight and a complete audit trail. The final GRADE judgements remain accountable expert decisions. Each one should be linked to the underlying evidence and accompanied by a clear reason.

If the rating cannot be explained, it is not ready for scrutiny.

Take home

• A meta-analysis provides an effect estimate. GRADE assesses how much confidence to place in it.
• Certainty is judged separately for every important outcome.
• Randomised evidence usually starts at high certainty and non-randomised evidence at low certainty. The final rating depends on the evidence, not the starting point.
• Risk of bias, inconsistency, indirectness, imprecision and publication bias can lower certainty.
• A serious problem usually means downgrading once. A very serious problem usually means downgrading twice.
• GRADE is structured judgement, not arithmetic. Every judgement needs a reason and a full audit trail.
• The approach to all five domains should be planned before the results are known. Every final judgement needs a reason and a full audit trail.
• Certainty informs a recommendation or reimbursement decision. It does not determine one.

References

1. NHS England. Evidence review: Siltuximab for idiopathic multicentric Castleman disease in adults. Searches conducted 27 May 2022. https://www.england.nhs.uk/wp-content/uploads/2023/11/2124-eveidence-report-siltuximab-castleman-disease.pdf
2. Schünemann HJ, Higgins JPT, Vist GE, Glasziou P, Akl EA, Skoetz N, Guyatt GH. Chapter 14: Completing ‘Summary of findings’ tables and grading the certainty of the evidence. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.5. Cochrane; 2024. https://training.cochrane.org/handbook/current/chapter-14
3. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. https://doi.org/10.1136/bmj.39489.470347.AD
4. Guyatt GH, Oxman AD, Sultan S, et al. GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. Journal of Clinical Epidemiology. 2013;66(2):151-157. https://doi.org/10.1016/j.jclinepi.2012.01.006
5. Schünemann HJ, Vist GE, Higgins JPT, et al. Chapter 15: Interpreting results and drawing conclusions. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.5. Cochrane; 2024. https://training.cochrane.org/handbook/current/chapter-15
6. National Institute for Health and Care Excellence. Pyelonephritis (acute): antimicrobial prescribing. Evidence review. NICE guideline NG111. October 2018. https://www.nice.org.uk/guidance/ng111/evidence/evidence-review-pdf-6545799181
7. National Institute for Health and Care Excellence. Developing NICE guidelines: the manual. PMG20. Published 31 October 2014; updated 23 October 2025. https://www.nice.org.uk/process/pmg20
8. NHS England. Methods: national clinical policies. Published 18 May 2026; updated 19 May 2026. https://www.england.nhs.uk/long-read/methods-national-clinical-policies/
9. Zorginstituut Nederland. Beoordeling Stand van de Wetenschap en Praktijk 2023. 11 April 2023. https://www.zorginstituutnederland.nl/documenten/2023/04/11/beoordeling-swp-2023
10. Medicinrådet. Metodevejledning for Medicinrådets vurdering af flere lægemidler inden for samme sygdomsområde (behandlingsvejledning). Version 2.0. Approved 20 June 2024. https://filer.medicinraadet.dk/media/nsmanhvx/metodevejledning-for-medicinradets-behandlingsvejledninger-vers-2-0.pdf
11. Institute for Quality and Efficiency in Health Care. General Methods. Version 7.0. 19 September 2023. https://www.iqwig.de/en/methods/methods-paper/