How Patient Reported Outcome Measures (PROMs) can help prioritisation of elective surgery and value

1.0 Introduction

On March 11, 2020, the World Health Organization (WHO) declared the novel coronavirus disease 2019 (COVID-19) a global pandemic, which classifies the outbreak as an international emergency[1]. At the time of writing, COVID-19 has been reported in 212 countries and infected over 3,582,233 people and resulted in 248,558 deaths[2].

In the UK, the national emergency response included releasing hospital capacity to allow the redeployment of staff to care for patients with Covid-19 resulting in the temporary suspension of most elective surgery that has especially affected patients requiring elective surgery including major joint replacement. The British Orthopaedic Association has produced a BOAST guideline to help guide management of orthopaedic patients during the pandemic. This states ‘It is acknowledged that, during the coronavirus pandemic, surgeons and patients will……..need to balance optimum treatment of a patient’s injury or condition against clinical safety and resources. The BOA supports reasoned pragmatic decision-making in these extraordinary circumstances……’. However, the guideline makes no reference to objective criteria for prioritising patients requiring ‘elective’ surgery nor the conditions precedent to considering this to have become ‘urgent’.

This article considers the role of Patient Reported Outcome Measures (PROMs) as a tool to objectively support the prioritisation of elective orthopaedic surgery (including post Covid-19)[3] as well as to identify best practice and potential under-performance. Together, the approach I will describe has the potential to drive greater adoption of value based healthcare by purchasers.

2.0     Background

Patient Reported Outcome Measures

PROMs measure a patient’s health status or health-related quality of life at a single point in time. This data is collected through short questionnaires completed by the patient. This information is collected before and after a procedure and provides an indication of the outcomes or quality of care. In the UK, this information has been collected by all providers of National health Service (NHS) funded care for certain elective surgery procedures since April 2019.[4]

There are currently two types of PROMs questionnaires in general use – generic and condition specific. EQ-5D is a standardized instrument developed in Europe for measuring generic health status and the one most often used by the NHS. It is a descriptive, preference-based health related quality of life measure with one question for each of the five dimensions that include mobility, self-care, usual activities, pain/discomfort and anxiety/depression. The answers enable patients to be allocated one of 243 unique health states or for their responses to be converted into EQ-5D index utility score anchored at 0 for death and 1 for perfect health[5]. It has been widely used in population health surveys, clinical studies, economic evaluation and in routine outcome measurement in the delivery of operational healthcare and is currently used alongside condition-specific in the NHS.

Condition-specific instruments are most commonly used in the NHS to monitor disability and outcome in patients undergoing elective orthopaedic surgery. These include the Oxford Hip score developed in 1996 followed by the Oxford Knee and Oxford Shoulder score in 1998. These were designed and developed by researchers within the Health Services Research Unit, part of the Nuffield Department of Population Health at the University of Oxford, and include twelve descriptive items that can be scored from 0 to 4 to give a maximum score of 48.

In the UK, providers are required to collect and submit PROMs data to NHS England for patients undergoing primary elective hip and knee replacement funded by the public sector. Most independent providers also collect this data for these elective surgical procedures with some extending this to cataract surgery and cosmetic surgery. NHS Digital publish an overview of the latest PROMs data once every quarter that includes[6]:

• An overview of the latest available PROMs data for every public and independent hospital undertaking relevant NHS-funded elective procedures;

• A summary of quality data;

• A score comparison spreadsheet tool with the latest available data for providers and clinical commissioning groups;

• Aggregate number of patients who have improved, unchanged or worsened based on the pre- and post- operative questionnaires for each procedure and measure;

• Participation Linkage, which shows numbers of participants, linked episodes and questionnaire issue and response rates for each procedure and measure;

• Time series data for each procedure and measure.

Impact of Covid-19 on elective surgery

The rapidly spreading outbreak imposes an unprecedented burden on the effectiveness and sustainability of healthcare systems. Acute challenges include the exponential increase in emergency department (ED) visits and inpatient admission volumes, in conjunction with the impending risk of health care workforce shortage due to viral exposure, respiratory illness, and logistical issues due to the widespread closure of school systems[7].

Subsequent to the WHO declaration, the United States Surgeon General proclaimed a formal advisory to cancel elective surgeries at hospitals due to the concern that elective procedures may contribute to the spreading of the coronavirus within facilities and use up medical resources needed to manage a potential surge of coronavirus cases[8]. 

In the UK, the elective surgery waiting list already stood at 4.4 million patients before the outbreak, the highest figure since the referral to treatment pathway began in 2007. Doctors have postponed more than 2 million operations after non-emergency surgery was cancelled for at least three months to free up beds for coronavirus patients.

The mounting backlog of procedures could cost the NHS £3bn to work through and may require many of the 20,000 doctors and nurses who have returned to the health service to stay on once the Covid-19 crisis has been brought under control.[9] The Royal College of Surgeons believes the NHS will need a five-year strategy to deal with a “mountain” of an elective waiting list following the coronavirus outbreak[10]. As a result, patients are being deprived of surgical access, with uncertain loss of function and risk of adverse prognosis as a collateral effect of the pandemic. Surgical services need a contingency plan for maintaining surgical care in an ongoing or post-pandemic phase[11].

Current guidance on resuming elective surgery

In the US, the American College of Surgeons (ACS) has issued a call to prioritise appropriate resource allocation during the coronavirus pandemic as it relates to elective invasive procedures[12]. The ACS bulletin made the following recommendation: ‘Each hospital, health system, and surgeon should thoughtfully review all scheduled elective procedures with a plan to minimize, postpone, or cancel electively scheduled operations, endoscopies, or other invasive procedures until we have passed the predicted inflection….’. Importantly, this recommendation ‘does not reflect on a presumed imperative to cancel all elective surgical cases across the United States’[13]. The guidance does not, however, offer an objective basis for determining which patients justify the risk to merit undergoing elective surgery. Instead, it categorises patients to a risk group according to procedure. The category titled ‘Elective (discretionary) >3 months’ combines cosmetic surgery, bariatric surgery, joint replacement sports surgery, vasectomy/tubal ligation and infertility procedures.

In the UK, the Royal College of Surgery has published recommendations on the recovery of surgical services during and after Covid-19[14]. Under the heading ‘Assessing surgical workload and patient population’, the guidance states under the heading ‘patient prioritisation’: ‘There should be clear prioritisation protocols that reflect local and national needs, alongside availability of local resources’. It also states: ‘NHS England is undertaking gap analysis on data from all trusts in England to estimate delays in referrals to secondary care compared to last year. This information will be helpful for planning a more efficient delivery of surgical services’. The College suggests that the risk to the surgical patient should include a combined assessment of the real risk of proceeding and the real risk of delay: ‘Plans for triage should avoid blanket policies but rather rely on a day-to-day, data-driven assessment of the changing risk-benefit analysis, taking into account expert clinical opinion’[15]. However, there is no mention of what objective and comparable data is currently available other than perhaps a patient’s waiting time. It can be argued that health function would be a far more rational, data driven and ultimately ethical basis for prioritisation of this cohort of patients at a time when the availability of resources is constrained.

Value based healthcare (VBHC)

All healthcare systems today are under pressure to spend their resources wisely and efficiently. Though great improvements have been achieved by strategies to enhance cost-effectiveness and performance of healthcare services within the last 20 years, an OECD report on “Wasteful Spending in Health” (2017) presented alarming data on inappropriate care and wasted resources with estimations ranging from a conservative 10% up to 34% of expenditures. [16] The European Union Expert Panel on Effective Ways of investing in Health in 2019 defined VBHC as ‘a comprehensive concept built on four value-pillars: appropriate care to achieve patients’ personal goals (personal value), achievement of best possible outcomes with available resources (technical value), equitable resource distribution across all patient groups (allocative value) and contribution of healthcare to social participation and connectedness (societal value).[17]

Value-based pricing is a system whereby healthcare providers are paid on the quality of care they provide rather than the number of healthcare services they give or the number of patients they treat. Value-based pricing may give patients access to better treatments at lower cost[18]. U.S. healthcare systems have been gradually heading towards the use of value-based payment models. Roughly 30% of healthcare payments in 2016 in hospital and outpatient clinic settings were paid through a non fee-for-service payment model. Reimbursement for physician services appears to be transitioning rapidly from volume to value-based payment. Alternatives to Medicare, Medicaid, and commercial insurer fee-for-service - including Medicare Advantage, Medicaid managed care organizations, Medicare accountable care organizations and certain bundled payment programs in the commercial sector are predicted to account for close to 60% of physician practices' revenues by 2019

3.0        Insights from PROMS data

PROMs, especially condition-specific questionnaires, may offer clinicians and those funding care some helpful and objective insights into a patient’s health status, the propensity for surgeons and provider organisations to intervene with surgery compared with their peers, and improvement in health function following surgery with statistically significant benchmarks.

There are three main key performance indicators (KPIs) with two sub-measures:

• Average Baseline score

• Average Follow Up score

+ % patients achieving Maximum Follow Up score

• Average Health Gain

+ % patients reporting negative Health Gain

Risk-adjustment should be considered, though it can be argued that with sufficiently large datasets, the effect for comparative purposes is inconsequential. For example, in relation to major joint replacement, the impact of risk adjustment for the indicators listed above is likely to be minimal as long as the PROM is used only for patients undergoing primary arthroplasty and not revision surgery, and when the volumes are sufficiently large. Nevertheless, risk-adjustment can if necessary be undertaken using the model developed for NHS England.

Funders looking to engage provider organisations with PROMs most often focus on Health Gain as a measure of ‘value’, though they should do so with some caution. Patients whose surgery is delayed – and therefore report on average a lower Baseline score – report on average a higher Health Gain than those with a higher Baseline score[21], but at the cost of a lower than average Follow Up score. So whilst it would appear that their Health Gain is greater, patients self-report a lower than average health function following surgery most likely as a result of their delayed treatment. This poses a clinical and ethical quandary in relation to the timing of surgical intervention – how early is too early (over-servicing) and how late is too late (resulting in sub-optimal treatment outcome).

4.0        Recommendations

a) PROMs KPIs

Individuals, provider organisations and funders should monitor a standard set of KPIs in relation to PROMs that enables legitimate comparison and the identification of both statistically significant superior performance and under-performance. A table could for example include the following metrics for each procedure type, using data from the previous 12 months and previous 36 months (to identify direction of travel) with national benchmarks published by NHS England:

• Av. Baseline score

• Av. Follow Up score

• % reporting Health loss (-ve Health Gain)

• % reporting maximum Follow Up score

• Av. Health Gain

The Baseline PROM score can be easily completed by patients remotely and electronically in order to prioritise those requiring elective surgery according to their current health function, rather than simply relying on the length of time they have been on a waiting list. Patients can be prompted on a regular basis – for example every six months – to repeat the questionnaire via an automated email or SMS prompt if they have not proceeded to surgery.

The Follow Up questionnaire is usually completed six, nine and/or twelve months later. The information collected should ideally be made available to patients via their own personal online account created by the PROMs administrator. This should include a time series setting out their own results as well as helpful resources and advice to support conservative treatments and self-care prior to and following surgery.

b) Indication and prioritisation for surgery

The Baseline score provides an objective assessment of an individual’s health function assuming it is completed correctly and without bias. This can therefore be used by payers and providers in the prioritisation for elective surgery across a large number of patients at a time when resources have become over-stretched.

In some cases, individual surgeons and provider organisations may report a statistically significant variation of their average Baseline score (two standard variations above the mean) in patients undergoing surgery. This may reflect a higher than average propensity to proceed to surgery compared with peer groups who may try more conservative treatments. This should prompt further review by Medical Directors and payers alike.

c) Monitoring effectiveness of surgery (value)

From a value-based healthcare perspectives, a Health Gain result within tolerance limits represents effective care. However, from the patient’s perspective, they are looking to achieve as good a health function, that is their Follow Up score following their surgery, as possible.

Each of these can be reported using a funnel plot to identify statistically significant outliers according to the volume of procedures undertaken – see Figure 1.

Figure 1. Funnel plot – average Health Gain v Procedure Volume (dummy data)

The effectiveness of surgery should therefore ideally be determined using both these key performance metrics. These can be both individually reported to identify both positive and negative statistical outliers and can also be represented according to a scattergram using published average scores (by NHS England) as per Figure 2, to report and benchmark the performance of individual surgeons within a provider group and/or of providers within a payer’s network.

Figure 2. Scattergram – average Health Gain v Follow Up score (dummy data)

Superior performance is suggested by those reporting both better than benchmark average Health Gain and Follow Up scores i.e. top right-hand quadrant, whilst conversely those in the lower left-hand quadrant may suggest under-performance compared with peers. However, the outcome of treatment is influenced not only by the surgeon but also by all those involved in the multi-disciplinary care of the patient. The patient’s clinical pathway includes care delivered prior to admission (such as hip schools and physical therapy) as well as following discharge. There may also be other confounding factors such as unrelated injuries following discharge that may affect the Follow Up score. Where a patient reports a negative Health Gain (typically c.3-4% of hip and knee replacement episodes), this should be reported as an Adverse Event and investigated to identify any and act upon addressable reasons related to their surgery.

Organisations should have in place systems to identify statistically significant negative outliers (two standard deviations below the mean) based on a minimum number of completed (linked) episodes (ideally a minimum of 20) and have a rigorous process for investigation and, if necessary, intervention as part of their clinical governance and safety framework. Whilst the underlying drivers of superior and under-performance may not be immediately apparent today, it is likely that AI and the analysis of large datasets will have a role to play in identifying patients who are at high risk of reporting negative Health Gain - enabling organisations to intervene - as well as to identify likely factors resulting in superior outcomes that can be systematised and hard-wired into clinical and operational processes.

5.0        Conclusion

In conclusion, PROMS can be used to:

• Inform prioritisation decision for elective surgery during and post-Covid-19;

• Identify potential over-servicing by individual surgeons and providers;

• Identify and learn from surgeons and providers delivering superior patient outcomes;

• Identify and investigate potential under-performance by individual surgeons and providers;

• Help payers commission value-based healthcare based on objective measures.

I have 3 recommendations for payers and provider organisations in relation to monitoring PROMs:

• Agree and monitor a standard set of meaningful PROMs KPIs to enable legitimate benchmarking of surgeons and providers;

• Use the Baseline score of condition-specific PROMs to inform prioritisation decisions regarding elective surgery when healthcare resources become over-stretched (and to identify potential over-servicing);

• Drive value based healthcare purchasing (and pricing) by assessing the effectiveness of surgeons and providers through analysis of both Health Gain and Follow Up scores using funnel plots and scattergrams to identify (and learn from) superior performers as well as to identify and investigate potential under-performance.