“When selecting a patient-centered outcome…for a large clinical trial…, we must agree what a clinically meaningful difference is.”

Image: J. P. Rathmell.

Image: J. P. Rathmell.

Patients consent to undergoing surgery in the hope that the procedure will decrease pain and disability and/or improve prognosis and quality of life. Optimizing the entire perioperative trajectory should contribute to high-quality patient outcomes and is the aim of perioperative medicine.

In this issue of Anesthesiology, Shulman et al.1  have undertaken the huge task of trying to identify what constitutes a meaningful change in disability—from the patient’s perspective—after surgery. The same authors previously demonstrated that a brief 12-item disability questionnaire, the World Health Organization’s Disability Assessment Schedule (WHODAS 2.0), can accurately capture changes in disability after surgery and thus can be used as a valid primary endpoint for perioperative clinical trials.2  WHODAS 2.0 has been internationally validated, can be completed by the patient in 5 min, and is widely used in all areas of medicine. The level of disability is measured on six dimensions: cognition, mobility, self-care, interpersonal relationships, work and household roles, and participation in society. It can be used on paper, online, and in a telephone interview; should the patient be incapacitated, a proxy (caregiver) can complete the questionnaire.

The goal of perioperative research is to identify and fill existing knowledge gaps. However, only high-quality clinical studies can tell us what really works—or what does not—and provide meaningful guidance for daily practice to help us achieve the best patient outcomes. Because large perioperative clinical trials are costly and time-consuming, designers must carefully consider the selection of their primary study endpoint. Until recently, many randomized, controlled trials in perioperative medicine used one or more indicators of pathophysiology, such as postoperative myocardial injury, lesions on cerebral magnetic resonance imaging (occult stroke), acute kidney injury, or postoperative cognitive dysfunction as primary outcome. Although such study endpoints are clearly important—and often associated with symptoms and patient complaints—each of these can also be present without any sign of new disability of decreased quality of life. In contrast, when we ask the patient to report changes in disability and quality of life after anesthesia and surgery, the results are considered patient-centered: They matter to patients and impact their daily lives. Patient-reported changes in disability are the end result of all perioperative pathophysiologic and psychologic effects on the patient’s well-being and daily functioning. When a therapy directed at the primary pathophysiology is also associated with potentially serious side-effects—for example, anticoagulant therapy—patient-reported outcomes such as WHODAS 2.0 may be better able to capture the combined net treatment result in terms of benefits and harms. Designers of large trials therefore increasingly avoid composite pathophysiology outcomes, such as myocardial injury, cardiac death, or stroke, which only capture events and disease states as the primary endpoint. Instead, the focus is on patient-centered outcomes, with composites as important secondary endpoints.

The majority of data for the current analysis by Shulman et al. came from the REstrictive versus LIbEral Fluid Therapy in Major Abdominal Surgery (RELIEF) trial, aimed to bring clarity in the restricted versus liberal perioperative fluids debate.3  In that study, 3,000 high-risk patients undergoing major abdominal surgery randomly received either a restrictive or liberal intravenous-fluid regimen up to 24 h after surgery. Disability-free survival at 1 yr was the primary study outcome. There was no difference in disability after 1 yr, although more kidney injury was seen in the restrictive fluids group.

What Is a “Meaningful” Change?

When selecting a patient-centered outcome as primary endpoint for a large clinical trial in perioperative medicine, we must agree what a clinically meaningful difference is. Standardization of patient-centered outcomes—informed by studies such as the one by Shulman et al.—may greatly improve the reliability and comparability of future perioperative trials and may help researchers to decide on efficient sample sizes.

Only the patient can decide on what defines a meaningful change in disability. The authors cleverly combined the patient’s opinion regarding the success of surgery with the changes in disability as measured with WHODAS 2.0; in this way they could anchor the abstract disability scores to the perceived benefit or harm from the surgery and arrive at the minimal clinically important difference in disability. In addition, they used statistical distribution-based methods to verify the observed minimal clinically important disability difference. With the WHODAS 2.0 score (converted to a 0 to 100% scale), a 5% change in disability turned out to be clinically meaningful from the patient perspective. The authors were also able to determine that a disability score less than 16% represents a patient acceptable symptom state (i.e., a score below which patients still consider themselves healthy and not disabled). In contrast, a disability score above 35% indicates important disability. Figure 1 shows how such changes in disability after anesthesia and surgery might be interpreted. Knowing these two disability anchor values and the minimal clinically important difference of 5% can help us design better perioperative studies with disability as patient-centered primary outcome.

Fig. 1.

Minimal clinically important difference in disability and patient-acceptable symptom state. The trajectories of six hypothetical patients are indicated by the arrows. Patient disability change arrows: Patient 1 had moderate preoperative disability (World Health Organization’s Disability Assessment Schedule [WHODAS]: 42); the surgery neither improved nor worsened disability (WHODAS: 45). Patient 2 had preoperative disability (WHODAS: 42); the patient considers the surgery a success, and postoperatively there is no longer any disability (WHODAS: 9). Patient 3 also had preoperative disability (WHODAS: 49); although the surgery improved disability, postoperatively there was still some disability (WHODAS: 26). Patient 4 was preoperatively free of disability (WHODAS: 6), but developed postoperative complications with new disability (WHODAS: 45). Patient 5 was disability-free preoperatively (WHODAS: 5) and remained so after surgery (WHODAS: 9). Patient 6 had major preoperative disability (WHODAS: 65); although there was clinically important improvement after surgery, she remained severely disabled (WHODAS: 57).

Fig. 1.

Minimal clinically important difference in disability and patient-acceptable symptom state. The trajectories of six hypothetical patients are indicated by the arrows. Patient disability change arrows: Patient 1 had moderate preoperative disability (World Health Organization’s Disability Assessment Schedule [WHODAS]: 42); the surgery neither improved nor worsened disability (WHODAS: 45). Patient 2 had preoperative disability (WHODAS: 42); the patient considers the surgery a success, and postoperatively there is no longer any disability (WHODAS: 9). Patient 3 also had preoperative disability (WHODAS: 49); although the surgery improved disability, postoperatively there was still some disability (WHODAS: 26). Patient 4 was preoperatively free of disability (WHODAS: 6), but developed postoperative complications with new disability (WHODAS: 45). Patient 5 was disability-free preoperatively (WHODAS: 5) and remained so after surgery (WHODAS: 9). Patient 6 had major preoperative disability (WHODAS: 65); although there was clinically important improvement after surgery, she remained severely disabled (WHODAS: 57).

A strength of the present work is that it only used data from patients undergoing anesthesia and surgery because the minimal clinically important difference might be different in specific patient groups or diseases. Ideally, the researchers would have had access to large numbers of data from unselected surgical patients of all ages. As the authors acknowledge, the data were dominated by older patients undergoing abdominal surgery, often for cancer (patients from the RELIEF study). This might have introduced some bias. To tackle this problem, they performed sensitivity analyses by comparing minimal clinically important differences in women versus men, older versus younger patients, patients with or without malignancy, as well as for different types of surgery. None of these sensitivity analyses resulted in large deviations from the 5% minimal clinically important difference observed in the combined group. Not unexpected, for patients with a higher rate of baseline disability, minimal clinically important difference estimates were slightly higher (i.e., they needed a larger improvement to be clinically meaningful). Consenting patients in clinical trials are not always representative of the entire population and may represent better prognostic risk categories and health-literate patients. The authors addressed this limitation by adding patients from two observational cohorts (one of which is still enrolling patients) where there was no previous selection based on inclusion criteria for a clinical trial.

When a study’s primary outcome is based on patient self-report, missing data can become a nightmare for the researchers. If data are missing because patients are not well and unable to return questionnaires, there is a considerable risk of biased results. When data are missing at random, sophisticated statistical modeling techniques (multiple imputation) can give best estimates for the missing value. The authors performed extensive missing data analyses that did not alter their estimate for minimal clinically important difference. What might go wrong when large amounts of data are missing can be seen in a recent study on changes in disability after free surgical care by the organization Mercy Ships in Madagascar.4  The authors used WHODAS 2.0 preoperatively (face-to-face) and by telephone 3 months after surgery interview to measure postoperative changes in disability. Unfortunately, only 44% of patients could be reached postoperatively. All patients in this subgroup of responders reported a significant reduction in their disability score (from 8% to 1%). By the new criteria derived by Shulman et al., there was no new disability after surgery in any patient. However, because patients lost to follow-up were younger, had longer hospital stays, and were more likely to have experienced postoperative complications, the change in disability was likely skewed toward the better outcomes.

Patient-reported outcomes are here to stay. The authors should really be complemented for this important work. Agreeing on what constitutes a meaningful clinically important difference in a patient-reported outcome will directly impact the sample sizes needed in future perioperative randomized, controlled trials. Thanks to the work by Shulman et al., we now have reliable anchor points for postoperative disability: minimal disability, a state that most patients consider acceptable (less than 16% disability on a 0 to 100 disability scale), the level above which patients definitely have moderate disability that negatively impacts their daily life (35%), and a minimal clinically important difference in disability of 5%, which informs decision making, both for clinical trials and quality improvement initiatives. I would definitely consider that a maximum result.

Competing Interests

Dr. Kalkman has received funding from the European Commission’s Horizon 2020 framework for a Precommercial Procurement project No. 727534, “Nightingale” (remote wireless patient monitoring): http://www.nightingale-h2020.eu.

References

1.
Shulman
MA
,
Kasza
J
,
Myles
PS
: .
Defining the minimal clinically important difference and patient-acceptable symptom state score for disability assessment in surgical patients.
Anesthesiology
.
2020
;
132
:
1362
70
2.
Shulman
MA
,
Myles
PS
,
Chan
MT
,
McIlroy
DR
,
Wallace
S
,
Ponsford
J
: .
Measurement of disability-free survival after surgery.
Anesthesiology
.
2015
;
122
:
524
36
3.
Myles
PS
,
Bellomo
R
,
Corcoran
T
,
Forbes
A
,
Peyton
P
,
Story
D
,
Christophi
C
,
Leslie
K
,
McGuinness
S
,
Parke
R
,
Serpell
J
,
Chan
MTV
,
Painter
T
,
McCluskey
S
,
Minto
G
,
Wallace
S
;
Australian and New Zealand College of Anaesthetists Clinical Trials Network and the Australian and New Zealand Intensive Care Society Clinical Trials Group
: .
Restrictive versus liberal fluid therapy for major abdominal surgery.
N Engl J Med
.
2018
;
378
:
2263
74
4.
White
MC
,
Randall
K
,
Alcorn
D
,
Greenland
R
,
Glasgo
C
,
Shrime
MG
: .
Measurement of patient reported disability using WHODAS 2.0 before and after surgical intervention in Madagascar.
BMC Health Serv Res
.
2018
;
18
:
305