Home › Answers

Answers

The questions that come up in almost every project, answered directly. If your question is not here, ask it — the answer usually takes one email.

What is the difference between real-world data and real-world evidence?

Real-world data is data collected outside a controlled trial. Real-world evidence is the conclusion produced when that data is analysed to answer a specific question. Data becomes evidence only through a design and an analysis that someone else can scrutinise.

The distinction matters commercially, because "we have access to real-world data" is often presented as though it were the same as having evidence. It is not. A registry extract is an input. What an assessment body evaluates is the study built on top of it: the question, the design, the assumptions, and how honestly the limitations are reported.


When will an HTA body accept real-world evidence instead of a randomised trial?

Most readily where randomisation is impossible or unethical, where the question concerns natural history, epidemiology or current standard of care, or where a single-arm trial needs an external comparator. Observational data is rarely accepted as a substitute for a feasible head-to-head trial.

In practice the acceptance gradient looks like this. Uncontested: disease burden, treatment patterns, resource use, characterising the population the technology will actually be used in. Usually accepted with scrutiny: long-term outcomes beyond trial follow-up, generalisability from a trial population to routine care, and safety in subgroups the trial could not size for. Contested: comparative effectiveness where the comparator group is constructed rather than randomised — accepted when the alternative is no evidence at all, and judged on how well the confounding has been handled.

What moves a package from the third category to the second is almost never a more sophisticated model. It is a clearer estimand, a design that pre-empts the obvious biases, and a bias analysis showing how large an unmeasured confounder would have to be to explain the result away.


What is the EU Joint Clinical Assessment and who does it apply to?

The Joint Clinical Assessment (JCA) is an EU-level assessment of a health technology's clinical evidence, introduced by Regulation (EU) 2021/2282. It applied first, from 12 January 2025, to new cancer medicines and advanced therapy medicinal products, extends to orphan medicinal products from 2028 and to all new medicines from 2030.

The JCA replaces duplicated clinical assessment across member states. It does not replace national decisions: pricing, reimbursement and any economic evaluation stay national, and each member state draws its own conclusion from the joint clinical report. Selected high-risk medical devices and in-vitro diagnostics are covered by a separate joint work stream.

The practical consequence for a developer is timing. The clinical dossier is submitted in parallel with the EMA application, which pulls a great deal of evidence work earlier than companies were used to — and leaves very little room to generate anything new once the scope is set.


Why does the Joint Clinical Assessment produce so many PICOs?

Each member state can specify the population, comparator and outcomes that match its own clinical practice. Because standard of care differs across Europe, one JCA's consolidated scope can contain many distinct PICO combinations, each of which the dossier must be able to address.

This is the single most consequential planning problem the Regulation created. A trial designed against one comparator can find itself facing a scope that names several, some of which were never studied directly. The answers then have to come from indirect comparison or from real-world data — both of which need to have been anticipated, because neither can be assembled in the weeks between scoping and submission.

The mitigation is unglamorous: map the plausible national comparators early, decide which PICOs you will be able to answer directly, and build the evidence for the rest deliberately rather than discovering the gap at scoping.


What makes Nordic health registries useful for real-world evidence?

National coverage, a unique personal identifier linking health, prescription and socioeconomic records for an entire population, and decades of continuous follow-up with very little loss. That combination answers questions about rare outcomes and long-term effects that most data sources cannot.

Denmark, Sweden, Norway, Finland and Iceland all maintain population registers of this kind, with national patient, prescription, cause-of-death and cancer registers alongside a large number of disease-specific clinical quality databases. For a study that needs twenty years of follow-up on an unselected population, there is little comparable anywhere.

The trade-offs are real and should be stated in any protocol. Clinical granularity is limited — a diagnosis code is not a phenotype. Registration practice, coding standards and register coverage change over calendar time, which can look exactly like a trend. And access runs through approvals and secure environments that add months to a timeline and restrict what can be exported.


What is target trial emulation?

A design discipline: first write down the randomised trial you would run if it were possible — eligibility, treatment strategies, assignment, start of follow-up, outcome, analysis — then build the observational study to match it as closely as the data allows.

Its value is that every departure from the hypothetical trial becomes explicit and has to be justified. Most of the classic design failures in observational research are departures that were never noticed: follow-up that starts at a different moment than eligibility is assessed, treatment strategies that are not actually distinguishable in the data, outcomes ascertained differently between groups.

It is not a statistical method and it does not fix unmeasured confounding. It fixes the errors that no amount of modelling can rescue afterwards.


What is immortal time bias?

Immortal time bias occurs when a period during which a participant could not, by definition, have had the outcome is counted as exposed follow-up — usually because exposure is defined by something that happens after follow-up begins. It systematically favours the treated group.

The textbook example: classify patients as "treated" if they ever fill a prescription during follow-up, and start their follow-up at diagnosis. Every treated patient must have survived long enough to fill it. The untreated group has no such requirement, and the treatment appears to extend survival even if it does nothing at all.

The fix is design, not adjustment: align the moment of eligibility, the moment of treatment assignment and the start of follow-up — or use a design, such as a cloning or landmark approach, that handles the misalignment explicitly.


What is an external control arm?

A comparator group built from data outside the trial — a registry, historical trial data, or routine care records — used when a single-arm trial has no internal comparator. Its credibility rests on comparability of patients, of outcome measurement, and on how transparently residual confounding is quantified.

Assessors ask a predictable set of questions about these, and they are worth pre-empting. Were the external patients eligible for the trial by the trial's own criteria? Were outcomes defined and ascertained the same way, at the same intervals? Does the external data come from the same calendar period, given how standard of care shifts? What happens to the estimate under plausible unmeasured confounding?

An external control that answers those clearly is often accepted as supportive. One that is silent on them tends not to be.


When should a statistical analysis plan be written?

Before the analysis is run, and ideally before the analyst has seen outcome data. A plan written afterwards cannot do the one thing that matters most: show that the reported result was not selected from many that were tried.

For observational studies this is more important than for trials, not less, because the number of defensible analytic choices is much larger. Exposure definition, washout, covariate set, functional form, handling of competing risks — each has several reasonable options, and the space of results they generate is wide. Pre-specification, with a version-controlled document and a dated record of amendments, is what converts a defensible choice into a demonstrable one.


How long does a registry-based real-world study take?

The analysis is rarely the long part. Feasibility and design typically take four to eight weeks, data access and approvals anywhere from weeks to many months depending on the register and jurisdiction, and analysis and reporting a further two to four months.

PhaseTypical durationWhat drives the variation
Question framing and feasibility2–4 weeksHow clearly the decision is defined
Protocol and analysis plan3–6 weeksNumber of stakeholders reviewing
Data access and approvals1–9 monthsRegister, jurisdiction, data controller, ethics route
Analysis4–10 weeksData quality; number of pre-specified analyses
Report and publication4–12 weeksInternal review; journal turnaround

Two things compress this reliably: starting the approval process in parallel with protocol writing rather than after it, and settling the estimand early enough that the analysis does not get re-scoped once results start appearing.

Last reviewed: . These answers describe general practice and are not advice on a specific product or submission.

Have a question that is not on this page?

Send it. If it has a short answer you will get one, with no expectation of a project attached.