Defining the Minimal Clinically Important Difference and Patient-acceptable Symptom State Score for Disability Assessment in Surgical Patients

There is increasing recognition that clinical trials should measure outcomes that are important to patients.^1,2  The patient-centered outcomes subgroup of the Standardised EndPoints in Perioperative Medicine (StEP-COMPAC) working group has recommended that the 12-item World Health Organization Disability Assessment Schedule 2.0 (WHODAS) should be included as a measure of functional status in clinical trials. WHODAS has undergone extensive psychometric evaluation in a diverse surgical population³  and has since been used as a primary or secondary endpoint in several clinical trials and cohort studies.^4–7 

Key metrics of a patient-rated scale, such as WHODAS, include the minimal clinically important difference, and the patient-acceptable symptom state.⁸  The minimal clinically important difference is the smallest change in score on a given scale that corresponds to a meaningful change in clinical state from the patient’s perspective,^9,10  whereas the patient-acceptable symptom state refers to a threshold score on the scale beyond which patients consider themselves to be well.^11,12 

Until now we have used data and recommendations from nonsurgical populations^13,14  to estimate a change in WHODAS score consistent with the minimal clinically important difference (8%) and an absolute WHODAS score (at least 25%) consistent with significant disability.³  These estimates need to be confirmed in a surgical cohort. Furthermore, a WHODAS 25% cutoff refers to patients with at least moderate disability and therefore does not represent a patient-acceptable state.

The aim of this study was to determine the minimal clinically important difference and patient-acceptable symptom state for the WHODAS score. In addition, a secondary objective was to determine the WHODAS score that is consistent with at least moderate or clinically significant disability in a surgical population.

This study combined prospectively collected data from two previously published and one ongoing perioperative medicine studies, all measuring disability using WHODAS:

• (1) The REstrictive versus LIbEral Fluid Therapy in Major Abdominal Surgery (RELIEF) Study was an international multicenter randomized controlled trial that enrolled 2,983 patients undergoing major abdominal surgery to a restrictive or liberal fluid therapy regimen.⁴  The primary endpoint of RELIEF was disability-free survival 12 months after surgery.

• (2) The WHODAS validation study was an international multicenter observational study that enrolled 510 patients to confirm the psychometric properties of WHODAS in a diverse surgical cohort.³ 

• (3) The MeasurIng Disability After Surgery (MIDAS) study is an ongoing single-center registry that had included 868 patients at the time of this analysis (Alfred HREC 279/16). MIDAS enrolls all patients 70 yr of age or older having emergency or elective, cardiac and noncardiac surgery at a tertiary Australian hospital. Patients completed WHODAS before surgery and by telephone with a trained interviewer at 3 and 6 months after surgery. Patients were provided information pamphlets and were able to opt out of having their data included in the registry. Patients were excluded if they refused to complete WHODAS before surgery; if surgery was time-critical; if they had poor English comprehension; or if there was known or suspected cognitive impairment, current psychiatric disease, or substance abuse.

All three studies measured disability using the 12-item version of WHODAS (see Supplemental Digital Content, fig. S1, https://links.lww.com/ALN/C298, the 12-item self-administered WHODAS 2.0) before surgery (baseline) and at 3 and 6 months after surgery. Each item asks about how much difficulty the patient has had because of health problems in a specific functional domain over the past 30 days and is scored on a 5-point Likert scale: none = 0; mild = 1; moderate = 2; severe = 3; and extreme = 4. The total 12-item score, between 0 and 48, was then divided by 48 and multiplied by 100 to convert it, by linear transformation, to a percentage of the total possible score as previously described.³ 

Missing WHODAS items were handled according to the process described in the WHODAS manual.¹⁴  For a single missing item, the score for this item was imputed using the average item score. If more than one item was missed, the total score was classified as missing data.

There is no agreed method of sample size determination for minimal clinically important difference studies, although previous studies have used less than 200 patients. We included three studies from our research group that use WHODAS to measure disability, with a combined sample size of 4,361 patients. We considered that this large, surgically diverse population would provide an accurate estimate of the minimal clinically important difference.

No prescribed methodology exists for minimal clinically important difference estimation.^8,10,15  Experts recommend that multiple approaches using a combination of anchor and distribution-based methods with triangulation (averaging) of results is the optimal method of minimal clinically important difference estimation,^9,10  acknowledging that each method has its strengths and limitations.¹⁶ 

Distribution-based methods use various measures of statistical distribution to estimate the minimal clinically important difference.⁸  They are easy to perform and have been shown to correlate well with anchor-based estimates.¹⁷  The disadvantage of distribution-based methods is that they do not contain an external reference, or anchor, to a patient experience or clinical event. As such, experts recommend that they should primarily be used to confirm the findings of anchor-based methods.^10,18 

We used three distribution-based methods proposed by Myles et al.¹⁹ : 0.3 times the SD, the SEM, and 5% of the score range. The SEM was calculated as the SD multiplied by the square root of 1 minus the intraclass correlation coefficient.²⁰ 

Because the minimal clinically important difference is sensitive to different population groups and clinical scenarios, a range of minimal clinically important difference estimates may exist for a given patient-centered outcome measure depending on the context in which it is used.^10,15  We conducted sensitivity analyses by comparing the minimal clinically important differences in women versus men, in older versus younger patients, in patients with or without a history of malignancy, and for different types of surgery. We also stratified patients in the WHODAS validation cohort using a self-rated measure of health. This was achieved by splitting this cohort into three equal tertiles using patient’s preoperative EuroQol-5D visual analog scores.²¹ 

Anchor-based methods estimate the minimal clinically important difference by relating a change in patient-centered outcome score to a change in clinical scenario or a change on a patient-reported global rating scale.¹⁵  The global rating scales use Likert scales to rank the patient’s improvement or deterioration, with the minimal clinically important difference equating to the mean change of the patient groups that “improving a little bit” or “becoming a little worse.”¹⁵  A limitation of this approach is that retrospective patients reports are subject to recall bias¹⁰  with the patient’s current state influencing their rating more than their previous state.²² 

We estimated the minimal clinically important difference using two anchor-based methods. First, the WHODAS validation study asked patients to rate the change in their health status at 3 and 6 months after surgery using two questions:

• (1) Did your surgery improve your daily life?

• (2) Did you feel better following surgery?

These questions were answered on a 5-point Likert scale: +2 (strongly agree), +1 (tend to agree), 0 (neither agree nor disagree), −1 (tend to disagree), and −2 (strongly disagree). If patients answered both questions with 1 or −1, this would be considered to be consistent with a change equivalent to the minimal clinically important difference. In patients meeting these criteria, the absolute change in WHODAS score could then be used to calculate an estimate of the minimal clinically important difference.

Second, all three studies measured unexpected readmission to hospital. This endpoint is often used because it signifies a clinically significant deterioration in health. It is also a patient-centered outcome measure, because most patients do not want to be readmitted to hospital after surgery. We used unexpected readmission to hospital at 3 months as a further anchor-based confirmation of the minimal clinically important difference. The three distribution and two anchor-based estimates were then averaged to reach a final minimal clinically important difference estimate. Multiple imputation was also conducted to assess the impact of missing WHODAS scores (see Methods of missing data assessment and Supplemental Digital Content, tables S1 to S3, https://links.lww.com/ALN/C298, which describe multiple imputation methods and data).

The patient-acceptable symptom state score was estimated using data from the WHODAS validation study cohort. Patients who answered “tend to agree” or “strongly agree” to the question “Did your surgery improve your daily life?” were selected as being likely to have an acceptable symptom state with little to no significant disability. In these patients, the patient-acceptable symptom state score was estimated as the 75th centile of WHODAS scores at 3 and 6 months, a method employed in previous studies.^11,12,23  This estimate was then compared with the 75th centile of the entire study cohort.

In contrast, the WHODAS score consistent with clinically significant (at least moderate) disability was estimated by two methods. Andrews et al.¹³  estimated that the top 10% of WHODAS scores represents patients who are likely to have significant disability. We repeated this analysis in the entire study cohort, measuring the 90^th centile at 3 and 6 months after surgery. A second method of estimating significant disability was made using the 75th centile of WHODAS scores at 3 and 6 months in patients who had been unexpectedly readmitted to hospital within that time frame.

The data are presented as means ± SD or number (%) unless otherwise specified. The change in WHODAS scores from baseline to 6 months after surgery was compared with the paired Student’s t test. Between-group comparisons for nonparametric data were made using the Mann–Whitney U test or Kruskal–Wallis test for multiple groups.

Internal consistency and responsiveness testing were performed on 3-month WHODAS scores. The standardized response mean was calculated in patients in the WHODAS validation study who at 3 months tended to disagree or strongly disagreed that surgery improved daily life and tended to disagree or strongly disagreed that they felt better after surgery. The standardized response mean was calculated as the mean change in WHODAS score between baseline and 3 months, divided by the SD of the change.

All analysis were undertaken using SPSS v.25. A P value of less than 0.05 was considered statistically significant. There was no correction for multiple comparisons.

Patient demographics (table 1) are presented using the combined data of 4,361 patients from the RELIEF, WHODAS validation, and MIDAS studies. The mean age of patients was 67 (range, 18 to 103) years and 45% of patients were female. Most patients had an American Society of Anesthesiologists (Schaumburg, Illinois; ASA) Physical Status score of II (35%) or III (54%), and a high proportion of patients had a history of malignancy (49%), with 10% having local or distant metastases. The majority of surgery was elective (91%), and the most common type of surgery was general abdominal surgery (59%). Clinically significant disability was present in 21% of patients before surgery, and 12% of patients were admitted to intensive care after surgery.

Mean WHODAS scores, indicating more disability, tended to be higher in patients with a higher ASA Physical Status score (table 2), preoperatively and at 3 and 6 months after surgery (P < 0.0005 for trend at each time point). As patients recovered from surgery, mean WHODAS scores improved (became lower) from baseline to 6 months, with a mean difference of 1% (95% CI, 1 to 2%; P < 0.0005).

Distribution-based estimates were made using baseline WHODAS scores from 4,266 patients. The score for a single item was imputed for three patients. WHODAS scores were missing in 95 patients, with 81 of these from patients enrolled in the WHODAS validation study before WHODAS incorporation at baseline at that study site.³ 

The distribution-based estimates of the minimal clinically important difference (table 3) were similar for SD, SEM, and 5% of instrument range methods, with an average minimal clinically important difference estimate of 5% in the entire study population. Sensitivity analysis found that minimal clinically important difference estimates were slightly higher for subgroups with a higher rate of baseline disability, including patients with lower baseline EuroQol-5D visual analog scores and patients having neurosurgery or orthopedic surgery (table 3). However, minimal clinically important difference estimates were similar for men, women, older (at least 70 yr of age) and younger (less than 70 yr of age) patients, and patients with or without a history of malignancy (table 3).

Anchor-based estimates of the minimal clinically important difference, based on patient-reported changes in health status and 3-month hospital readmission, were similar to distribution-based estimates (table 4). The average of the mean differences from the five methods of anchor-based estimation was 5%. In contrast, the mean difference in WHODAS scores between baseline and 3 months in patients not readmitted to the hospital within 3 months of surgery was −1%.

Triangulating the average distribution-based minimal clinically important difference estimate (5%; table 3) and five anchor-based minimal clinically important difference estimates (table 4), we reached a final minimal clinically important difference estimate of 5%. Minimal clinically important difference estimates were similar when multiple imputation was conducted to account for missing data (see Supplemental Digital Content, tables S2 and S3, https://links.lww.com/ALN/C298, which are multiply imputed data set versions of tables 3 and 4). Using this minimal clinically important difference value to quantify the change in disability from baseline to 6 months after surgery, we concluded that 750 patients (21.4%) had a significant increase in disability, 1,060 patients (30.3%) had a significant decrease in disability, and 1,687 patients (48.2%) had no significant change in disability.

Cronbach’s α (internal consistency) of the WHODAS score was 0.90. The standardized response mean (responsiveness) of WHODAS was 0.5. The scaling properties are demonstrated in figure 1, with 30% of patients having a score of 0, and 84% of patients having a WHODAS score of less than 25% (table 5). Median WHODAS scores were significantly higher in patients who were readmitted within 3 or 6 months after surgery, compared with patients who were not (P < 0.0005 for both; see Supplemental Digital Content, table S4, https://links.lww.com/ALN/C298, which describes median WHODAS scores after surgery).

For patients selected as likely to have a patient-acceptable symptom state, the 75th centile for WHODAS scores was 17% (n = 255) at 3 months and 15% (n = 256) at 6 months after surgery, with an average estimate score of 16%. In contrast, the 75th centile for WHODAS scores in the entire study cohort was higher, being 19% at 3 months and 17% at 6 months after surgery (average 18%). Using a rounded patient-acceptable symptom state score estimate of less than 16% to define a disability-free population, the disability-free survival rate was 73.2% 6 months after surgery (see Supplemental Digital Content, table S5, https://links.lww.com/ALN/C298, which describes the proportion of patients with various levels of disability after surgery).

The WHODAS score representing clinically significant (at least moderate) disability was estimated by two methods: (1) the 90^th centile for WHODAS scores in the entire cohort, which was 38% at 3 months and 35% at 6 months after surgery, and (2) the 75th centile for WHODAS scores in patients readmitted within 3 or 6 months after surgery, which was 33% at 3 months and 33% at 6 months after surgery. Using these four WHODAS scores, the average estimated WHODAS score to represent clinically significant disability was 35%. Using this definition, 367 patients (10%) had clinically significant disability 6 months after surgery (see Supplemental Digital Content, table S4, https://links.lww.com/ALN/C298, which describes median WHODAS scores after surgery).

We estimate that the minimal clinically important difference for WHODAS in surgical patients is 5%. This means that when converting the WHODAS score to a percentage scale for patients having surgery, a change in WHODAS score of 5% or more is consistent with a clinically meaningful increase or decrease in their level of disability. In perioperative research, this value can be used to define a significant change in disability in response to a treatment and can also be used to define new disability in conjunction with an absolute WHODAS value above which significant disability is defined.

Distribution-based minimal clinically important difference estimates varied minimally in patients with higher or lower rates of preoperative disability but were otherwise consistent between different patient groups. Furthermore, repeated distribution and anchor-based estimates using multiple imputation methods for missing data were very similar to original estimates. It is therefore likely that the minimal clinically important difference estimate of 5% is an accurate assessment of the true minimal clinically important difference and generalizable to most surgical settings and patient populations.

This study confirmed the findings of the WHODAS validation study that patients with a higher ASA score tended to have higher WHODAS scores and the postoperative trajectory in WHODAS scores, with most patients tending to recover with time and WHODAS scores being lowest at 6 months after surgery (table 2).³  We estimated that a WHODAS score of less than 16% is consistent with a patient-acceptable symptom state and that a WHODAS score of at least 35% can be used to define a patient with clinically significant (at least moderate) disability. This differs from our previously published recommendation that clinically significant disability should be defined as a WHODAS score of at least 25% and that anyone with a WHODAS score of less than 25% could be considered disability-free. We believe these new definitions are more accurate and applicable to clinical practice and research for several reasons. The previous definition was based on data from the general population^13,14  rather than a surgical cohort, where a WHODAS scores of at least 25% was considered to be consistent with at least moderate disability. The problem with this definition is that it uses a single point from a continuous scale to create a binary outcome. It follows that the population considered to be disability-free then may have included some patients with mild disability. In contrast, the new definitions classify two populations: one that has an acceptable symptom state and can therefore be considered disability-free and another with clinically significant disability that is more likely to be associated with ongoing health problems and hospital readmissions. Patients that do not fit into either category (having a WHODAS score between 16 and 35%) can be considered as having mild disability.

The patient-centered outcomes subgroup of the StEP-COMPAC working group has recommended that the 12-item version of WHODAS be included to measure disability as a standard clinical trial endpoint. We recommend the following specific scoring definitions:

• Minimal clinically important difference: increase or decrease on WHODAS score of at least 5%

• Disability-free survival: alive with a WHODAS score less than 16%

• Clinically significant disability: WHODAS score of at least 35%

• New onset clinically significant disability: increase in WHODAS score of at least 5% to a final WHODAS score of at least 35%

This study has some limitations. This was a retrospective analysis of prospectively collected data from three studies, each with a distinct patient population. In particular, the RELIEF study represents a cohort of patients predominantly undergoing abdominal surgery, and the MIDAS study only includes patients 70 yr of age or older. It is therefore possible that the minimal clinically important difference and patient-acceptable symptom state estimates are biased toward these two populations. However, sensitivity analysis demonstrated that averaged distribution-based minimal clinically important difference estimates were similar in patients having different types of surgery and in patients with varying degrees of preoperative and postoperative disability, suggesting that the estimated minimal clinically important difference is generalizable rather than specific to the included cohorts.

We also used anchor questions from the initial WHODAS validation study. These questions were not specifically designed for this minimal clinically important difference and patient-acceptable symptom state analysis and also limited any anchor-based analysis to the smaller WHODAS validation population. However, we believe the anchor question has face validity for minimal clinically important difference and patient-acceptable symptom state estimation, and studies of this kind are frequently conducted in populations smaller than the WHODAS validation population. Further, the WHODAS validation cohort was diverse, including elective, emergency, cardiac, noncardiac, and day-stay surgery. Finally, the anchor-based estimates were very similar to the distribution-based estimates, further confirming the appropriateness of using these anchor questions.

In conclusion, we have determined the minimal clinically important difference and patient-acceptable symptom state for the WHODAS scale in surgical patients. We have proposed definitions that can be incorporated into future clinical research and audit in line with the proposed inclusion of WHODAS as a standardized measure of perioperative outcome.

Supported by a project grant from The Australian and New Zealand College of Anaesthetists (Melbourne, Victoria, Australia).

The authors declare no competing interests.