Real world evidence: why we patients need it, and how we can get it

Patients and advocates welcome the dawn of precision medicine. But we want more certainty about the benefits and the risks. So far these new drugs have been used by very few patients in small phase II trials, and the long-term data we have are mixed. There is no statistical certainty: there is single-case or small-group benefit, but only assumptions about overall efficacy. Nonetheless registration follows.

Attitudes towards the balance between faster access and greater certainty may differ between patient activists and between patients and experienced advocates. Having been both patient and advocate over the years, I try to balance the viewpoints.

It is important not to be fooled by surrogate endpoints such as Progression Free Survival. They are not about survival, but about delaying the certainty of progression. There is a rational case that Overall Survival and Quality of Survival are the only measures that make any sense for decision-making about standards of treatment. This implies that more rigorous evidence than is offered by phase II studies is needed. Hence the call for real world evidence.

However, as a patient advocate, I see no standard methodologies, no funding, no independence emerging in this area. This is not good for patients. The call for real world evidence is a headline answer to the loss of rigour, and like most headlines it is simplistic. It is not clear how clinically relevant evidence can be gathered and analysed in a manner that ensures completeness, quality, and freedom from bias.

The patient advocate should not accept second best, and we must stand up and say so. This rigour is what we have become used to and it is what we still need. Precision medicine may be providing a tumour response in a small number of patients, but is it a high quality of treatment? We just do not know.

The key element is the quality of survival. Assessing quality of life is an evolving area of research, moving from all-encompassing Quality-of-Life models towards targeted patient reported outcomes (PROs), developed with patient input, measured long term, with data gathered using new technologies. This helps us look at specific aspects of survival, including side effects, on a longitudinal and long-term basis. Putting this alongside Overall Survival, with both measures derived in a rigorous manner, can give us the real world evidence we need.

Yes, we want any patient who believes that a particular treatment would benefit them, and whose clinicians agree, to have access to that treatment as soon as possible. But at the same time we think that a treatment should only be made available if it has robust evidence that it offers holistic benefits to patients in the target patient group.

There is a fine boundary between these two stances, which is brought into sharp focus by the issue of payment. A payer will usually hold a position balanced between the scale of cost and the degree of benefit that the evidence shows can be expected. A healthcare system, hospital, or insurer will have analysts and health economists to advise them. They can refuse funding even when a treatment is licensed.

A patient who is funding treatment from their own resources or those of supporters may not be so sophisticated. Extending mortgages, cashing in pension funds, reaching out through crowd funding – we hear about all these financial routes to treatment. Suddenly there is a lottery – where is the boundary between degrees of evidential certainty and financial exploitation? Clinicians sit in the middle of this, keen to support their patients but reluctant to see them spend money on what may be undeliverable expectations.

To try and get back some certainties, a group of leading cancer societies, led by the European Organisation for Research and Treatment of Cancer (EORTC) and supported by many organisations including the European Society for Medical Oncology (ESMO), is proposing a move to put standards into appraising the real world clinical use of new treatments through ‘treatment optimisation’. Their proposal would employ methods proven in clinical research to develop robust evidence. In addition, such studies could allow structured testing of the best ways of using the new technology, including identifying minimum effective dose – a move that could save on costs.

Medicine is not a lottery and should not become one, but it is in danger of doing so. Real world evidence is an obvious thing to be doing, and many patients will wonder why it is not already being done. The question then becomes how to gather real world evidence in our clinical systems to a standard of completeness and rigour that compares with what we are used to? Various stakeholders in the world of precision medicine have come at this challenge in different ways, but no clear sense of how to do it emerges.

The people with the most to gain are patients, but we have had no meaningful voice. Even though the views of patients and of experienced patient advocates sometimes differ, one set of priorities comes through clearly: we must not deny access to those who may benefit when a new drug is approved, but we must build certainty that the safety and efficacy characteristics of that treatment are real in the longer term and, crucially, that the quality of life of patients taking that treatment reflects benefit.

Where is the problem? Perhaps surprisingly to some, patients generally expect that some kind of ‘virtuous’ feedback loop happens in medicine. They find it a shock when they discover that no such feedback loop exists.

The opposite page shows a suggested model of how such a thing could be brought about, using an imaginary scenario played out in Ruritania – an imaginary, relatively wealthy, western European state.

For me this scenario constructs a model that has the opportunity to deliver rigour, independence, absence of bias and completeness of data. It respects pharma innovation, serves the needs of healthcare systems, and above all identifies the value in terms that patients want.

The key that unlocks it is ‘ROAR’, the independent but funded Ruritanian Oncology Agent Research agency. The scenario illustrated in the model opposite certainly begs a whole range of further questions – it is an imaginary scenario after all. But is it really too much to ask all the stakeholders to come together and reach some form of agreement that can bring rigour and certainty back into the equation that patients are forever balancing? I will happily chair the meeting.

A proposed model for integrating real world evidence into the evaluation of new drugs

 

Step 1: Market access

A pharmaceutical company Runoutofink Laboratories Ltd submits its experimental anticancer drug canigetanib to the regulatory authorities for marketing approval, which is given on the basis of:

• Two phase II studies both demonstrating improved progression free survival (PFS) of 3 months in a genetically defined subset of patients with cancer xyz. The studies show that the drug was well tolerated.

Step 2: Reimbursement decision

The Ruritanian Health Technology Assessment (HTA) programme agrees funding terms with Runoutofink Laboratories Ltd for a 3-year period subject to a study of real world evidence.

Step 3: Real world evidence study

A real world evidence study is:

• Conducted by an independent academic research group, according to a protocol agreed between the researchers, Runoutofink Laboratories Ltd and the Ruritanian HTA body,
• Funded by a national fund, the Ruritanian Oncology Agent Research agency (ROAR), set up jointly between the healthcare system and the pharma industry. The healthcare system funds the treatment. Runoutofink Laboratories Ltd supplies the drug under usual market terms, but only to doctors who have adopted the protocol.

The protocol requires that:

• Any doctor wishing to prescribe the drug does so under the terms of the protocol and data gathering standards,
• Every patient provided with canigetanib accepts that their data will be part of the study
• Patients are asked to provide their own reports of outcomes using a smartphone or internet access (via voucher code).

The protocol:

• Determines a standard follow-up with details of data to be gathered and standards to be applied,
• Gives scope for tighter follow-up and emergency action at the clinician’s discretion,
• Indicates side-effect reporting and treatment, including dose reduction.

The key data point at patient death is confirmed by the national registry.

Step 4: The real world evidence tells us more information

The real world study gathers data from several thousand patients with cancer xyz and shows:

• A 6-week overall survival benefit compared with historical data extracted from the Ruritanian national registry.
• PFS in the study is shorter than in the trials, explained by the fact that the trial cohort was younger than the average patient.
• Side effects are generally well tolerated, but reports of grade 3 adverse events are more frequent – again a factor of trial population.
• Dose reduction has been shown to reduce these side effects with no discernable difference in tumour response or duration of response. However longer-term side effects appeared after 12 months on continuous treatment, even on reduced dose, which patients reported affected their quality of life,
• Patients on average withdrew from treatment 5 months before dying and they experienced an improvement in side effects and quality of life for a period after withdrawal.

Step 5: Results are fed back into the system to inform further research and adjust pricing

• Research: ROAR withdraws the real world protocol. A phase III randomised clinical trial comparing a reduced dose with standard dose canigetanib is designed and put in motion, funded by ROAR. It has an overall survival primary endpoint. ROAR is considering academic proposals for phase II studies of combinations with other therapies.
• Reimbursement: After 3 years the Ruritanian HTA body gives a further period of approval to canigetanib, but the healthcare system reduces funding for the full dose of the drug after its HTA health economics analysis takes the ‘real world’ response and previously unknown side effects issues into account.

We await the results of the dose reduction study.