Editor’s Perspective
What We Already Know about This Topic
  • Frailty is not uncommon in the surgical population, with reported incidences of 2 to 45%, depending on the assessment used and the population assessed

  • Frailty has been associated with adverse outcomes after a variety of surgical procedures, elective or emergent, and in different age groups

  • Frailty can be assessed with various approaches and measurement tools

What This Article Tells Us That Is New
  • The Hopkins Frailty Score (a phenotype model) and the Modified Frailty Index score (a measure of deficit accumulation) were both poor predictors of unexpected prolonged hospital stay and a composite of readmission and serious complications

Background

Frailty is associated with adverse postoperative outcomes, but it remains unclear which measure of frailty is best. This study compared two approaches: the Modified Frailty Index, which is a deficit accumulation model (number of accumulated deficits), and the Hopkins Frailty Score, which is a phenotype model (consisting of shrinking, weakness, exhaustion, slowness, and low physical activity). The primary aim was to compare the ability of each frailty score to predict prolonged hospitalization. Secondarily, the ability of each score to predict 30-day readmission and/or postoperative complications was compared.

Methods

This study prospectively enrolled adults presenting for preanesthesia evaluation before elective noncardiac surgery. The Hopkins Frailty Score and Modified Frailty Index were both determined. The ability of each frailty score to predict the primary outcome (prolonged hospitalization) was compared using a ratio of root-mean-square prediction errors from linear regression models. The ability of each score to predict the secondary outcome (readmission and complications) was compared using ratio of root-mean-square prediction errors from logistic regression models.

Results

The study included 1,042 patients. The frailty rates were 23% (Modified Frailty Index of 4 or higher) and 18% (Hopkins Frailty Score of 3 or higher). In total, 12.9% patients were readmitted or had postoperative complications. The error of the Modified Frailty Index and Hopkins Frailty Score in predicting the primary outcome was 2.5 (95% CI, 2.2, 2.9) and 2.6 (95% CI, 2.2, 3.0) days, respectively, and their ratio was 1.0 (95% CI, 1.0, 1.0), indicating similarly poor prediction. Similarly, the error of respective frailty scores in predicting the probability of secondary outcome was high, specifically 0.3 (95% CI, 0.3, 0.4) and 0.3 (95% CI, 0.3, 0.4), and their ratio was 1.00 (95% CI, 1.0, 1.0).

Conclusions

The Modified Frailty Index and Hopkins Frailty Score were similarly poor predictors of perioperative risk. Further studies, with different frailty screening tools, are needed to identify the best method to measure perioperative frailty.

Frailty denotes an increased vulnerability or a diminished capacity to cope with external stressors. The pathophysiology of this poorly characterized and complex clinical syndrome remains poorly understood, but it is often attributed to reduced physiologic reserve. Frailty has been associated with adverse outcomes after a variety of surgical procedures, elective or emergent, and in different age groups.1–4  The reported incidence of frailty among patients presenting for noncardiac surgery ranges from 2 to 13%, depending on the frailty assessment tool used and definition thresholds.5,6  The incidence of frailty is substantial in high-risk populations, including patients having emergency surgery (26%)7  and elderly patients having major orthopedic surgery (41%).8 

Initially, frailty measurement tools were constructed to predict mortality and other adverse events such as falls, disability, and hospitalization among community-dwelling elders.9–12  Such assessment tools have been extended to characterize frailty in the perioperative setting.4  However, the best method to assess perioperative frailty remains unknown,13  perhaps explaining why frailty is not routinely used for perioperative risk stratification.14 

There are two broad approaches to characterizing frailty.15  The first, and best studied, is the deficit accumulation model. This approach assumes that frailty occurs due to accumulation of multiple deficits across various domains.11  The Modified Frailty Index is a deficit accumulation model based on the proportion of comorbidities present in a patient.16,17  It is typically calculated with data from the patient’s medical record or from a perioperative clinical database. The second approach is the phenotype model, which defines frailty as a clinical syndrome, determined by quantitatively measuring five clinical features: shrinking, weakness, exhaustion, slowness, and low physical-activity level.12  These variables are rarely recorded during preoperative assessments and are thus not readily available in electronic records. A perioperative example is the Hopkins Frailty Score.18 

Both the Modified Frailty Index and the Hopkins Frailty Score predict adverse outcomes after surgery.1,5,18–22  It is unclear, however, which paradigm of frailty assessment better assesses perioperative risk. Because there is no gold standard to validate these frailty assessment tools, comparison of their abilities to predict adverse postoperative outcomes would be most useful. Thus, the primary aim of this study was to compare the Modified Frailty Index and the Hopkins Frailty Score on their ability to predict prolonged hospitalization, measured as the difference between actual and expected surgery-specific postoperative length of stay. Secondarily, we compared these two tools on their ability to predict the secondary outcome, a composite of 30-day readmission and various serious postoperative complications.

Adults presenting to the Preoperative Anesthesia Consultation and Evaluation Clinic at the Cleveland Clinic (Cleveland, Ohio) Main Campus were considered for inclusion into this prospective observational cohort study. Approval for this study was obtained from the Cleveland Clinic Institutional Review Board.

Patients scheduled for noncardiac surgery at our institution are initially triaged using a model adapted from the American College of Cardiology (Washington, D.C.) and American Heart Association (Dallas, Texas) guidelines based classification of surgical risk and an online health survey score, HealthQuest, developed by the Department of General Anesthesiology, Cleveland Clinic, Cleveland, Ohio.23  Using this process, about half of our surgical patients are triaged to be evaluated at the preoperative evaluation clinic (appendix 1).

Patients at the preoperative clinic are arbitrarily assigned to evaluation by registered nurses trained in preoperative evaluation or anesthesiology residents, both supervised by an attending anesthesiologist. Frailty assessments are not routinely conducted. Three of the preoperative nurses were trained to administer a study-specific questionnaire and collect data for calculation of the Hopkins Frailty Score. All patients presenting to these specific trained providers were included in the study. Because frailty assessment is a recommended part of preoperative assessment for patients, based on American College of Surgeons (Chicago, Illinois) best practice guidelines, written informed consent was waived by the Institutional Review Board. However, patients were given written information about the study and had the option not to participate (appendix 2). Verbal consent was obtained by the nurse who did the assessment.

Measurements

The Modified Frailty Index includes 11 variables (table 1). The occurrence of each variable, except New York Heart Association (New York, New York) functional classification, was determined based on International Classification of Diseases (9th or 10th revision)–based billing codes for each variable associated with the surgical hospitalization. These data were retrieved from the Cleveland Clinic Perioperative Health Documentation System registry once all patients were recruited. This registry captures perioperative data for all patients undergoing surgery at Cleveland Clinic. The functional classification was assessed by the interviewer during the preoperative visit and documented in our REDCap study database.

Table 1.

Components of Modified Frailty Index

Components of Modified Frailty Index
Components of Modified Frailty Index

The Hopkins Frailty Score includes five components, specifically shrinking, exhaustion, low physical activity, slowed walking speed, and weakness (fig. 1).12,18  Each component is given a score of 1 when present, and the cumulative score forms the Hopkins Frailty Score for each patient. Physical activity was measured based on the short version of the Minnesota Leisure Time Activity questionnaire.24  Grip strength was measured using the dominant hand with a hand-held dynamometer (Sammons Preston Rolyan, Jamar Technologies, USA). Weakness was defined as a reduction in grip strength based on sex and body mass index–based normative data. Similarly, slow walking speed was determined based on normative data for sex and height. An average of three consecutive measurements was used to determine the presence of slowed walking speed and a reduction in grip strength. Normative data–based cutoffs for walking speed and grip strength used in the Hopkins Frailty Score are reported in figure 1.

Fig. 1.

Assessment criteria for calculating Hopkins Frailty Score. Reprinted from Journal of American College of Surgeons, 210(6), Makary MA, Segev DL, Pronovost PJ, Syin D, Bandeen-Roche K, Patel P, Takenaga R, Devgan L, Holzmueller CG, Tian J, Fried LP, Frailty as a predictor of surgical outcomes in older patients, pages 901-8, Copyright (2010), with permission from Elsevier.

Fig. 1.

Assessment criteria for calculating Hopkins Frailty Score. Reprinted from Journal of American College of Surgeons, 210(6), Makary MA, Segev DL, Pronovost PJ, Syin D, Bandeen-Roche K, Patel P, Takenaga R, Devgan L, Holzmueller CG, Tian J, Fried LP, Frailty as a predictor of surgical outcomes in older patients, pages 901-8, Copyright (2010), with permission from Elsevier.

Close modal

The Hopkins Frailty Score assessments were entered into an iPad (Apple, Inc., USA) during the preoperative visit and transmitted to our online REDCap study database. They were thus not incorporated into the electronic medical record. Patients with a Hopkins Frailty Score of 3 or more are classified as frail.12,25  Among various cutoffs used to designate frailty based on the Modified Frailty Index, the score of 4 or higher out of 11 is most commonly reported.6,26  However, both scores were considered as continuous variables for our analyses.

The primary and secondary outcomes were calculated based on data retrieved from the Cleveland Clinic Perioperative Health Documentation System registry. Length of stay for our study was defined as the total number of postoperative days spent in the hospital. When patients were released from the hospital on the day of surgery, we assigned patient length of stay to be 1 day. Length of stay is influenced by two relevant non–patient-related factors: the type of surgery and hospital-related factors. To adjust for these confounders, our primary outcome—prolonged hospitalization—was defined as the difference between the actual and expected length of hospitalization (prolonged hospitalization = actual length of stay – expected length of stay). The actual length of stay was the true postoperative length of hospitalization for each patient in our cohort. The expected length of stay (hospital- and surgery-specific) for a patient was estimated based on the median length of postoperative hospitalization for all adults between January 1, 2010 and December 31, 2015 who had same procedure according to the Current Procedural Terminology procedure code at our institution.

The secondary outcome was a binary composite that included 30-day complications and 30-day readmission after the noncardiac surgery. Postoperative complications were defined a priori based on the postoperative morbidity survey described by Bennett-Guerrero et al.27  and the Agency for Healthcare Research and Quality reported postoperative complications (table 2) and were retrieved from the Cleveland Clinic Perioperative Health Documentation System registry using International Classification of Diseases (9th or 10th revision)–based codes for surgical admission or readmission within 30 days. Because all postoperative complications that might occur in frail patients are unknown, we included readmission as an outcome because it captures various serious events. The occurrence of 30-day readmission was captured from the Cleveland Clinic Perioperative Health Documentation System registry, which included readmission to any Cleveland Clinic Health System hospital.

Table 2.

Components of the Secondary Outcome Composite

Components of the Secondary Outcome Composite
Components of the Secondary Outcome Composite

Statistical Analysis

Primary Analysis.

For the first simplified description of the relationship between the Modified Frailty Index and Hopkins Frailty Score, we used the Pearson correlation coefficient; Fisher’s z transformation was used to report confidence limits on the correlation coefficient. The analysis described below addressed the primary aim of the study comprehensively.

For the primary analysis, the accuracy with which each frailty score predicts prolonged hospitalization was calculated using root-mean-squared prediction error (prediction error); that is, the average distance between the observed and predicted outcome. If a frailty score improves the accuracy with which we predict prolonged hospitalization, model residuals are smaller, thereby reducing the root-mean-squared prediction error. Controlling for bias imposed by a given sample, bootstrap resampling28  cross-validation techniques were used to compare the prediction errors between two linear regression models with prolonged hospitalization as the outcome and the two frailty indices as predictors.

Based on the original data set, we generated a bootstrap sample using sampling with replacement, randomly partitioning our sample into five equal subsamples and calculating five-fold cross-validated prediction errors for the Modified Frailty Index and Hopkins Frailty Score regression models. The ratios of root-mean-squared prediction error between the Modified Frailty Index and Hopkins Frailty Score were calculated as well. We repeated bootstrap sampling 10,000 times. Cross-validated prediction errors were averaged over the 10,000 bootstrap data sets for both the Modified Frailty Index and Hopkins Frailty Score models, with the lowest value indicating the best predictive model; 95% bootstrap confidence intervals were also reported.

To evaluate the primary aim of the study, which is to compare the abilities of the Modified Frailty Index and Hopkins Frailty Score to predict prolonged hospitalization, a 95% bootstrap confidence interval of prediction error ratio was constructed. If the confidence interval for this ratio did not include one, then a statistically significant difference in predictive ability could be claimed.

Two final linear regression models for the Modified Frailty Index and Hopkins Frailty Score scoring systems were fit using the entire data set. The model based two-sided Wald test P values were reported to measure univariate association between the frailty indices and the primary outcome (prolonged hospitalization), with significance criteria set at P < 0.05. Discrimination ability was estimated via coefficient of determination R2 (the proportion of the variance in outcome explained by the model; R2 indicates how well data fit a statistical model for the continuous outcome) for each model. Calibration of the two models was assessed by plotting predicted versus observed outcome (prolongation in hospitalization) averaged over patients with identical predicted values. Ideal calibration would be indicated by values close to the 45° line on a plot.

As a sensitivity analysis, we performed the primary analysis using log-transformed actual length of hospitalization as the primary outcome instead of prolonged hospitalization. Log transformation made the data more normally distributed.

Secondary Analysis.

To evaluate the secondary aim of the study, which is to compare the ability of the Modified Frailty Index and Hopkins Frailty Score to predict a composite outcome of 30-day complications and readmission, we used algorithms similar to those used for the primary analysis. Because the secondary outcome is a binary outcome, the two compared models were logistic regression models. R statistical software version 2.15.1 for 64-bit Microsoft Windows (R Foundation for Statistical Computing, Austria) and SAS software version 9.4 (SAS Institute, USA) were used for statistical analyses.

Sample Size and Power Analysis.

The sample size calculation was based on the primary aim of comparing the Modified Frailty Index and Hopkins Frailty Score models on their ability to predict prolongation in postoperative length of hospitalization after noncardiac surgery. For the purpose of sample size estimation, we used an F-test for comparing coefficients of determination R2 for two nested regression models. The significance level was set at 0.05 for the primary aim.

Hypothesizing that the R2 of the two models (coefficients of determination) would be 0.30 and 0.33 and assuming a standard deviation of 0.15, 1,000 patients would provide approximately 94% power for a two-sided test comparing the discrimination ability of the two models. Adjusting for three interim analyses at each 25% of the planned enrollment, a maximum of 1,190 patients were required.

Between March 2, 2015 to November 9, 2016, 1,182 adult patients presenting to the preoperative clinic at the Cleveland Clinic main campus were included in this study. The study was stopped when we had 1,226 surveys and believed 1,190 unique patients were enrolled; however, 38 surveys were incomplete, and 6 turned out to be duplicate patients.

We excluded patients with missing medical record numbers, patients who did not have surgery within 90 days of the preoperative visit, and patients with missing preoperative study assessments. We also excluded patients who had surgeries lasting less than 30 min. Finally, patients with incomplete outcome data for analysis were excluded. That left 1,042 unique patients in our analysis cohort (fig. 2).

Fig. 2.

Study cohort.

The demographic data, baseline characteristics, types of surgeries, and frailty measures are reported in table 3. The distribution of Hopkins Frailty Score and Modified Frailty Index values across the population is shown in figure 3. Based on the Hopkins Frailty Score, 184 (18%) patients were classified as frail. Based on a Modified Frailty Index cutoff of 4 or greater, 234 (23%) patients of our cohort were classified as frail. The median length of stay was 2 days (interquartile range: 1, 4), and the median prolongation in hospitalization (primary outcome) was −1 day (interquartile range: −2, 0; table 4). The secondary outcome occurred in 12.9% of the patients. The most commonly reported postoperative complication was renal injury, occurring in 7.0% of the patients, and 6.5% of the patients were readmitted within 30 days of the surgery (table 4). The Pearson correlation (95% Fisher’s z transformation confidence limits) between the Modified Frailty Index and Hopkins Frailty Score scores was 0.28 (95% CI, 0.22 to 0.33), indicating a weak positive linear relationship between the two frailty indices.

Table 3.

The Demographic and Baseline Characteristics, including Frailty Indices for the Study Population (N = 1,042)

The Demographic and Baseline Characteristics, including Frailty Indices for the Study Population (N = 1,042)
The Demographic and Baseline Characteristics, including Frailty Indices for the Study Population (N = 1,042)
Table 4.

Summary of the Postoperative Outcomes Given as Medians [First Quartile, Third Quartile] or N (%), as Appropriate (N = 1,042)

Summary of the Postoperative Outcomes Given as Medians [First Quartile, Third Quartile] or N (%), as Appropriate (N = 1,042)
Summary of the Postoperative Outcomes Given as Medians [First Quartile, Third Quartile] or N (%), as Appropriate (N = 1,042)
Fig. 3.

Distribution of Modified Frailty Index (MFI, A) and Hopkins Frailty Score (HFS, B) scores across the study cohort (N = 1,042).

Fig. 3.

Distribution of Modified Frailty Index (MFI, A) and Hopkins Frailty Score (HFS, B) scores across the study cohort (N = 1,042).

Close modal

Primary Analysis

The prediction error of the Modified Frailty Index model was smaller than the Hopkins Frailty Score model, their ratio being 1.01 (95% CI, 1.00 to 1.02), indicating a statistically significant but clinically unimportant difference between the frailty indices in prediction accuracy (table 5). Despite the Modified Frailty Index being a significantly better predictor than the Hopkins Frailty Score, the prediction error for prolonged hospitalization was high (more than 2 days) for both scores. Thus, the ability of either frailty model to predict prolonged duration of hospitalization was poor. Similarly, the discrimination ability estimated via coefficient of determination (R2) was poor for both the Hopkins Frailty Score (0.002) and the Modified Frailty Index (0.023). Both calibration plots deviated substantially from the ideal 45° line for observed versus predicted prolongation in hospitalization (fig. 4).

Table 5.

Summary of the Prediction Models’ Performance for Difference between Actual and Expected Hospital Length of Stay and Probability of Postoperative Complication/Readmission Composite (N = 1,042)

Summary of the Prediction Models’ Performance for Difference between Actual and Expected Hospital Length of Stay and Probability of Postoperative Complication/Readmission Composite (N = 1,042)
Summary of the Prediction Models’ Performance for Difference between Actual and Expected Hospital Length of Stay and Probability of Postoperative Complication/Readmission Composite (N = 1,042)
Fig. 4.

Calibration plots of Modified Frailty Index (MFI, A) and Hopkins Frailty Score (HFS, B) models (N = 1,042).

Fig. 4.

Calibration plots of Modified Frailty Index (MFI, A) and Hopkins Frailty Score (HFS, B) models (N = 1,042).

Close modal

The sensitivity analysis produced similar results. There was no difference between the Modified Frailty Index and Hopkins Frailty Score frailty assessments in prediction of log-transformed actual length of hospitalization. The bias corrected prediction error was 2.2 (95% CI, 2.1 to 2.2) days for both models (more than 2 days), and the estimated coefficient of determination R2 was 0.009 and 0.019, respectively.

Secondary Analysis

Table 5 also shows that the Modified Frailty Index and Hopkins Frailty Score were comparably bad at predicting the composite of 30-day readmission and postoperative complications, with the ratio of their prediction errors being 1.00 (95% CI, 0.99 to 1.01). Furthermore, the C statistic was 0.59 (95% CI, 0.54 to 0.64) for the Modified Frailty Index and 0.60 (95% CI, 0.55 to 0.65) for the Hopkins Frailty Score. The two measures thus predicted complications poorly and comparably so.

In patients presenting for elective noncardiac surgery, the Modified Frailty Index better predicted differences between actual and expected hospital duration than the Hopkins Frailty Score, but not by a clinically important amount. Both scores were comparably bad at predicting a composite of 30-day postoperative complications and/or readmission. Thus, neither was a useful predictor of hospital duration or 30-day postoperative complications and/or readmission.

In a similar study, Cooper et al.8  evaluated 415 orthopedic surgery patients using the two paradigms of frailty. They used the Fried Index to measure the phenotype paradigm, which is similar to the Hopkins Frailty Score, but used a more comprehensive 40-point scale to assess the deficit accumulation paradigm. As in our study, Cooper et al.8  found that correlation between two paradigms of frailty was marginal. Neither was a good predictor of readmissions or postoperative complications, although frailty was associated with postoperative hospitalization exceeding 5 days (relative risk, 3.1). Other studies evaluating noncardiac surgical patients compared different deficit accumulation measurement tools but did not show clinical superiority of any one tool over the other.6,29,30 

Our most surprising result, and therefore the most interesting, is that neither measure of frailty was a helpful predictor of length of hospitalization or 30-day complications and/or readmission. Both frailty scores resulted in errors exceeding 2 days in predicting surgery-specific length of hospitalization. Similarly, the C statistics for 30-day readmission and/or postoperative complications with our two assessment strategies were only 0.59 and 0.60 (with 0.5 indicating predictions no better than chance). Although statistically significant, these low values indicate that predictive ability was too weak to be clinically useful.

In contrast to our results, other investigations report that frailty, quantified with either deficit accumulation or phenotype models, is associated with prolonged hospital stay.1,8,20,31–33  Two recent meta-analyses34,35  report statistically significant odds ratios in a majority of the studies evaluating the impact of frailty on length of stay. However, substantial interstudy heterogeneity prevented pooling of study level effect estimates for length of stay. These results need to be interpreted with caution, because most studies use length-of-stay calculations that were not surgery-specific.1,8,31  For example, Flexman et al.1  used the National Surgical Quality Improvement Program database to evaluate 53,080 patients who had surgery for degenerative spine diseases. They found that the Modified Frailty Index was associated with prolonged hospitalization (odds ratio 1.88 for every 10% increase in the Modified Frailty Index; 95% CI, 1.81 to 1.99). However, their patients with the highest Modified Frailty Index scores had the most complex multilevel spine surgery. Hence, adjusting for the type of surgery as a confounder markedly decreased their odds ratio to 1.27.

Our study cohort was heterogeneous with patients having various noncardiac surgeries, each with its own expected length of postoperative hospitalization. We accounted for type of surgery as a confounder by separately estimating the expected duration of hospitalization for each type of surgery, based on the median length of stay for that procedure at our institution. Our approach improves on previous investigations that did not account for the type of surgery and used semiarbitrarily defined prolonged hospitalization. For instance, Partridge et al.33  defined prolonged hospitalization as a postoperative stay of more than 12 days after arterial vascular surgery. In addition, in contrast to others who reported the strength of association between frailty and length of stay using odds ratios,1,8,20,31–33  we measured the prediction error using root-mean-squared prediction error, which better quantifies clinical relevance. Our more robust analysis shows that there is little relationship between frailty and the prolongation in expected duration of hospitalization after noncardiac surgery, with an error exceeding 2 days. Our sensitivity analysis, evaluating the relationship between frailty and absolute length of hospitalization without accounting for type of surgery, also revealed a prediction error greater than 2 days for both indices.

Many investigators report significant associations between measures of frailty and postoperative complications,1,5,6,8,16,19,20,29,36–40  but only some report C statistics,6,29,30,33,36,37,39  which quantify clinical importance. Reported C statistics range from 0.53 to 0.68, with most being between 0.60 and 0.65,6,29,30,39  which are similar to our findings. A statistically significant odds ratio34,35  does not imply predictive ability or discriminative classification, making measures of discrimination such as the C statistic critical.41  The available evidence therefore suggests only weak associations between frailty and postoperative complications, presumably because many other factors contribute to postoperative morbidity.

Because frailty assessments are rarely included in administrative or clinical registries, frailty represents a potential unknown confounding factor. The fact that both deficit accumulation and phenotype paradigms of frailty are poor predictors of hospital duration and complications is therefore reassuring from a research perspective. Furthermore, various risk evaluation schemes effectively include components measured in the deficit accumulation paradigm. For example, the American College of Surgeons National Surgical Quality Improvement Program Risk Calculators is a surgery-specific risk assessment tool that shares many patient variables with the Modified Frailty Index.42  Similarly, the Risk Stratification Index models, objectively derived by identifying various International Classification of Diseases (9th revision)–based billing codes that predict adverse outcomes, include many that might be considered indicators of frailty.43  Therefore, omitting deficit accumulation or phenotype–based measurements of frailty may not introduce unobserved confounding in otherwise well adjusted registry analyses.

The incidence of frailty in our cohort is comparable with6,38  or higher than1,5,26  other large studies evaluating perioperative frailty. By virtue of recruiting patients triaged to preoperative clinic evaluation, we selected a higher-risk population, potentially increasing power. However, it remains possible that these frailty indices better discriminate postoperative adverse outcomes in a different surgical cohort than ours. Frail patients have better postoperative outcomes when they undergo surgery at centers with a high volume of frail patients.44  The Cleveland Clinic Main Campus is a high-volume center, and many of our patients are frail; for example, about 20% were frail in our current sample, and 80% were American Society of Anesthesiologists physical status III or IV. It seems likely that institutional experience with frail and high-risk patients in general mitigated the impact of frailty on postoperative outcomes.

The difference between the findings of our study and previous literature can be attributed, at least in part, to our robust study design, which analyzes prediction errors and discriminative ability of the models. Prior studies evaluating the relationship between frailty and postoperative outcomes commonly report statistically significant odds ratio which overlook the score’s clinical utility.

A limitation of our study is that the elements of the Modified Frailty Index and postoperative outcomes were obtained from an institutional registry and thus could be influenced by factors affecting capture of variables in administrative databases.45  Moreover, 30-day readmission might be underestimated because 15% of study patients were from out-of-state. However, missing readmissions will not change the validity of our conclusions because the comparisons were within patients. Although formal assessments of frailty obtained for our study were not included in the electronic medical record, clinicians could of course make their own assessments of frailty and modify perioperative management in response, thereby affecting outcome.

Because there is no gold standard for measuring frailty in the perioperative context, it is not possible to evaluate the accuracy of the Modified Frailty Index and Hopkins Frailty Score. Thus, predictive validation is an important aspect in all perioperative frailty studies. Many investigators report statistically significant associations between both of these frailty scores and adverse postoperative outcomes, thereby erroneously reporting good predictive potential in clinical practice.16,17  However, poor discriminative ability, as described in our study, makes them poor clinical tools for identifying patients at risk. Poor reliability or reproducibility of these tests may also be an important source of error in our study. Interrater reliability has not been reported for either the Modified Frailty Index or the Hopkins Frailty Score. Having a pretrained group of test givers, as in our study, presumably improves interrater reliability of a test. Moreover, objective tests such as the ones we used usually have high interrater reliability, although we did not qualify this aspect of reliability.

We tested only one measurement system from each frailty assessment paradigm, and it remains possible that other methods outperform the two we used. The ones we selected, however, are well validated in nonoperative contexts and are easy to administer perioperatively. The Modified Frailty Index, for example, is the most widely studied deficit accumulation model, and it predicts adverse postoperative outcomes in large administrative databases5,16,17  and small prospective studies.20,46  One may argue that the Modified Frailty Index consists of only 11 variables, which do not cover all the described domains of frailty. However, a total of 11 elements is similar to the 10-variable threshold suggested for developing a stable frailty assessment model.47  The Modified Frailty Index is also comparable to other deficit accumulation models including the risk analysis index and Ganapathi Index for predicting postoperative outcomes.6,36  Additionally, the Fried Index, which is the basis for the Hopkins Frailty Score, has been validated in a perioperative setting.18,21  Recently, Kapoor et al.39  compared the Fried Index with the self-reported Late Life Function and Disability Instrument and found them to be comparable in perioperative settings. It thus seems unlikely that our results would much differ had we selected other measures of frailty.

The poor predictive ability of Hopkins Frailty Score and Modified Frailty Index may reflect the fact that neither was developed to assess surgical risk. In fact, the deficit-accumulation model of frailty was initially developed using variables from the Canadian Study of Health and Aging as a proxy measure for aging, mortality, and other adverse events in community-dwelling elderly.10,11  Furthermore, neither the Hopkins Frailty Score nor the Modified Frailty Index has been directly validated against their comprehensive original models. Novel frailty assessment tools developed specifically for surgical risk assessment might provide better predictive accuracy.

In summary, the Hopkins Frailty Score and Modified Frailty Index were comparably poor predictors of prolonged hospitalization and a composite of readmission and serious complications. Despite previous reports of frailty being a useful marker of perioperative risk, neither of two well established measures was predictive in our patients. As a corollary, omitting measures of frailty seems unlikely to introduce substantial unobserved confounding in otherwise well adjusted registry analyses. Because ours was a single-center study evaluating two specific frailty assessment tools, further studies, especially with different frailty screening tools, are needed to identify the best method to measure perioperative frailty and to establish its clinical utility.

Research Support

Support was provided solely from institutional and/or departmental sources.

Competing Interests

The authors declare no competing interests.

1.
Flexman
AM
,
Charest-Morin
R
,
Stobart
L
,
Street
J
,
Ryerson
CJ
: .
Frailty and postoperative outcomes in patients undergoing surgery for degenerative spine disease.
Spine J
.
2016
;
16
:
1315
23
2.
Beggs
T
,
Sepehri
A
,
Szwajcer
A
,
Tangri
N
,
Arora
RC
: .
Frailty and perioperative outcomes: A narrative review.
Can J Anaesth
.
2015
;
62
:
143
57
3.
Bagnall
NM
,
Faiz
O
,
Darzi
A
,
Athanasiou
T
: .
What is the utility of preoperative frailty assessment for risk stratification in cardiac surgery?
Interact Cardiovasc Thorac Surg
.
2013
;
17
:
398
402
4.
Buigues
C
,
Juarros-Folgado
P
,
Fernández-Garrido
J
,
Navarro-Martínez
R
,
Cauli
O
: .
Frailty syndrome and pre-operative risk evaluation: A systematic review.
Arch Gerontol Geriatr
.
2015
;
61
:
309
21
5.
Ali
TZ
,
Lehman
EB
,
Aziz
F
: .
Modified frailty index can be used to predict adverse outcomes and mortality after lower extremity bypass surgery.
Ann Vasc Surg
.
2018
;
46
:
168
77
6.
Hall
DE
,
Arya
S
,
Schmid
KK
,
Blaser
C
,
Carlson
MA
,
Bailey
TL
,
Purviance
G
,
Bockman
T
,
Lynch
TG
,
Johanning
J
: .
Development and initial validation of the risk analysis index for measuring frailty in surgical populations.
JAMA Surg
.
2017
;
152
:
175
82
7.
McIsaac
DI
,
Moloo
H
,
Bryson
GL
,
van Walraven
C
: .
The association of frailty with outcomes and resource use after emergency general surgery: A population-based cohort study.
Anesth Analg
.
2017
;
124
:
1653
61
8.
Cooper
Z
,
Rogers
SO
Jr
,
Ngo
L
,
Guess
J
,
Schmitt
E
,
Jones
RN
,
Ayres
DK
,
Walston
JD
,
Gill
TM
,
Gleason
LJ
,
Inouye
SK
,
Marcantonio
ER
: .
Comparison of frailty measures as predictors of outcomes after orthopedic surgery.
J Am Geriatr Soc
.
2016
;
64
:
2464
71
9.
Rockwood
K
,
Stadnyk
K
,
MacKnight
C
,
McDowell
I
,
Hébert
R
,
Hogan
DB
: .
A brief clinical instrument to classify frailty in elderly people.
Lancet
.
1999
;
353
:
205
6
10.
Mitnitski
AB
,
Mogilner
AJ
,
MacKnight
C
,
Rockwood
K
: .
The mortality rate as a function of accumulated deficits in a frailty index.
Mech Ageing Dev
.
2002
;
123
:
1457
60
11.
Mitnitski
AB
,
Mogilner
AJ
,
Rockwood
K
: .
Accumulation of deficits as a proxy measure of aging.
ScientificWorldJournal
.
2001
;
1
:
323
36
12.
Fried
LP
,
Tangen
CM
,
Walston
J
,
Newman
AB
,
Hirsch
C
,
Gottdiener
J
,
Seeman
T
,
Tracy
R
,
Kop
WJ
,
Burke
G
,
McBurnie
MA
;
Cardiovascular Health Study Collaborative Research Group
: .
Frailty in older adults: Evidence for a phenotype.
J Gerontol A Biol Sci Med Sci
.
2001
;
56
:
M146
56
13.
Hubbard
RE
,
Story
DA
: .
Patient frailty: The elephant in the operating room.
Anaesthesia
.
2014
;
69
:
26
34
14.
Eamer
G
,
Gibson
JA
,
Gillis
C
,
Hsu
AT
,
Krawczyk
M
,
MacDonald
E
,
Whitlock
R
,
Khadaroo
RG
: .
Surgical frailty assessment: A missed opportunity.
BMC Anesthesiol
.
2017
;
17
:
99
15.
Walston
JD
,
Bandeen-Roche
K
: .
Frailty: A tale of two concepts.
BMC Med
.
2015
;
13
:
185
16.
Velanovich
V
,
Antoine
H
,
Swartz
A
,
Peters
D
,
Rubinfeld
I
: .
Accumulating deficits model of frailty and postoperative mortality and morbidity: Its application to a national database.
J Surg Res
.
2013
;
183
:
104
10
17.
Wahl
TS
,
Graham
LA
,
Hawn
MT
,
Richman
J
,
Hollis
RH
,
Jones
CE
,
Copeland
LA
,
Burns
EA
,
Itani
KM
,
Morris
MS
: .
Association of the modified frailty index with 30-day surgical readmission.
JAMA Surg
.
2017
;
152
:
749
57
18.
Makary
MA
,
Segev
DL
,
Pronovost
PJ
,
Syin
D
,
Bandeen-Roche
K
,
Patel
P
,
Takenaga
R
,
Devgan
L
,
Holzmueller
CG
,
Tian
J
,
Fried
LP
: .
Frailty as a predictor of surgical outcomes in older patients.
J Am Coll Surg
.
2010
;
210
:
901
8
19.
Arya
S
,
Kim
SI
,
Duwayri
Y
,
Brewster
LP
,
Veeraswamy
R
,
Salam
A
,
Dodson
TF
: .
Frailty increases the risk of 30-day mortality, morbidity, and failure to rescue after elective abdominal aortic aneurysm repair independent of age and comorbidities.
J Vasc Surg
.
2015
;
61
:
324
31
20.
Cloney
M
,
D’Amico
R
,
Lebovic
J
,
Nazarian
M
,
Zacharia
BE
,
Sisti
MB
,
Bruce
JN
,
McKhann
GM
,
Iwamoto
FM
,
Sonabend
AM
: .
Frailty in geriatric glioblastoma patients: A predictor of operative morbidity and outcome.
World Neurosurg
.
2016
;
89
:
362
7
21.
Revenig
LM
,
Canter
DJ
,
Taylor
MD
,
Tai
C
,
Sweeney
JF
,
Sarmiento
JM
,
Kooby
DA
,
Maithel
SK
,
Master
VA
,
Ogan
K
: .
Too frail for surgery?: Initial results of a large multidisciplinary prospective study examining preoperative variables predictive of poor surgical outcomes.
J Am Coll Surg
.
2013
;
217
:
665
70.e1
22.
Shin
JI
,
Kothari
P
,
Phan
K
,
Kim
JS
,
Leven
D
,
Lee
NJ
,
Cho
SK
: .
Frailty index as a predictor of adverse postoperative outcomes in patients undergoing cervical spinal fusion.
Spine (Phila Pa 1976)
.
2017
;
42
:
304
10
23.
Parker
BM
,
Tetzlaff
JE
,
Litaker
DL
,
Maurer
WG
: .
Redefining the preoperative evaluation process and the role of the anesthesiologist.
J Clin Anesth
.
2000
;
12
:
350
6
24.
Taylor
HL
,
Jacobs
DR
Jr
,
Schucker
B
,
Knudsen
J
,
Leon
AS
,
Debacker
G
: .
A questionnaire for the assessment of leisure time physical activities.
J Chronic Dis
.
1978
;
31
:
741
55
25.
Afilalo
J
,
Lauck
S
,
Kim
DH
,
Lefèvre
T
,
Piazza
N
,
Lachapelle
K
,
Martucci
G
,
Lamy
A
,
Labinaz
M
,
Peterson
MD
,
Arora
RC
,
Noiseux
N
,
Rassi
A
,
Palacios
IF
,
Généreux
P
,
Lindman
BR
,
Asgar
AW
,
Kim
CA
,
Trnkus
A
,
Morais
JA
,
Langlois
Y
,
Rudski
LG
,
Morin
JF
,
Popma
JJ
,
Webb
JG
,
Perrault
LP
: .
Frailty in older adults undergoing aortic valve replacement: The FRAILTY-AVR study.
J Am Coll Cardiol
.
2017
;
70
:
689
700
26.
Louwers
L
,
Schnickel
G
,
Rubinfeld
I
: .
Use of a simplified frailty index to predict Clavien 4 complications and mortality after hepatectomy: Analysis of the National Surgical Quality Improvement Project database.
Am J Surg
.
2016
;
211
:
1071
6
27.
Bennett-Guerrero
E
,
Welsby
I
,
Dunn
TJ
,
Young
LR
,
Wahl
TA
,
Diers
TL
,
Phillips-Bute
BG
,
Newman
MF
,
Mythen
MG
: .
The use of a postoperative morbidity survey to evaluate patients with prolonged hospitalization after routine, moderate-risk, elective surgery.
Anesth Analg
.
1999
;
89
:
514
9
28.
Efron
B
,
Tibshirani
RJ
:
An Introduction to the Bootstrap
, 1st edition.
Boca Raton, Florida
,
Chapman and Hall/CRC
,
1994
29.
Amrock
LG
,
Neuman
MD
,
Lin
HM
,
Deiner
S
: .
Can routine preoperative data predict adverse outcomes in the elderly?: Development and validation of a simple risk model incorporating a chart-derived frailty score.
J Am Coll Surg
.
2014
;
219
:
684
94
30.
Kovacs
J
,
Moraru
L
,
Antal
K
,
Cioc
A
,
Voidazan
S
,
Szabo
A
: .
Are frailty scales better than anesthesia or surgical scales to determine risk in cardiac surgery?
Korean J Anesthesiol
.
2017
;
70
:
157
62
31.
Joseph
B
,
Zangbar
B
,
Pandit
V
,
Fain
M
,
Mohler
MJ
,
Kulvatunyou
N
,
Jokar
TO
,
O’Keeffe
T
,
Friese
RS
,
Rhee
P
: .
Emergency general surgery in the elderly: Too old or too frail?
J Am Coll Surg
.
2016
;
222
:
805
13
32.
Melin
AA
,
Schmid
KK
,
Lynch
TG
,
Pipinos
II
,
Kappes
S
,
Longo
GM
,
Gupta
PK
,
Johanning
JM
: .
Preoperative frailty risk analysis index to stratify patients undergoing carotid endarterectomy.
J Vasc Surg
.
2015
;
61
:
683
9
33.
Partridge
JS
,
Fuller
M
,
Harari
D
,
Taylor
PR
,
Martin
FC
,
Dhesi
JK
: .
Frailty and poor functional status are common in arterial vascular surgical patients and affect postoperative outcomes.
Int J Surg
.
2015
;
18
:
57
63
34.
Lin
HS
,
Watts
JN
,
Peel
NM
,
Hubbard
RE
: .
Frailty and post-operative outcomes in older surgical patients: A systematic review.
BMC Geriatr
.
2016
;
16
:
157
35.
Watt
J
,
Tricco
AC
,
Talbot-Hamon
C
,
Pham
B
,
Rios
P
,
Grudniewicz
A
,
Wong
C
,
Sinclair
D
,
Straus
SE
: .
Identifying older adults at risk of harm following elective surgery: A systematic review and meta-analysis.
BMC Med
.
2018
;
16
:
2
36.
Esses
G
,
Andreopoulos
E
,
Lin
HM
,
Arya
S
,
Deiner
S
: .
A comparison of three frailty indices in predicting morbidity and mortality after on-pump aortic valve replacement.
Anesth Analg
.
2018
;
126
:
39
45
37.
Gani
F
,
Cerullo
M
,
Amini
N
,
Buettner
S
,
Margonis
GA
,
Sasaki
K
,
Kim
Y
,
Pawlik
TM
: .
Frailty as a risk predictor of morbidity and mortality following liver surgery.
J Gastrointest Surg
.
2017
;
21
:
822
30
38.
Isharwal
S
,
Johanning
JM
,
Dwyer
JG
,
Schimid
KK
,
LaGrange
CA
: .
Preoperative frailty predicts postoperative complications and mortality in urology patients.
World J Urol
.
2017
;
35
:
21
6
39.
Kapoor
A
,
Matheos
T
,
Walz
M
,
McDonough
C
,
Maheswaran
A
,
Ruppell
E
,
Mohamud
D
,
Shaffer
N
,
Zhou
Y
,
Kaur
S
,
Heard
S
,
Crawford
S
,
Cabral
H
,
White
DK
,
Santry
H
,
Jette
A
,
Fielding
R
,
Silliman
RA
,
Gurwitz
J
: .
Self-reported function more informative than frailty phenotype in predicting adverse postoperative course in older adults.
J Am Geriatr Soc
.
2017
;
65
:
2522
8
40.
Sridharan
ND
,
Chaer
RA
,
Wu
BB
,
Eslami
MH
,
Makaroun
MS
,
Avgerinos
ED
: .
An accumulated deficits model predicts perioperative and long-term adverse events after carotid endarterectomy.
Ann Vasc Surg
.
2018
;
46
:
97
103
41.
Pepe
MS
,
Janes
H
,
Longton
G
,
Leisenring
W
,
Newcomb
P
: .
Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker.
Am J Epidemiol
.
2004
;
159
:
882
90
42.
Bilimoria
KY
,
Liu
Y
,
Paruch
JL
,
Zhou
L
,
Kmiecik
TE
,
Ko
CY
,
Cohen
ME
: .
Development and evaluation of the universal ACS NSQIP surgical risk calculator: A decision aid and informed consent tool for patients and surgeons.
J Am Coll Surg
.
2013
;
217
:
833
42.e13
43.
Chamoun
GF
,
Li
L
,
Chamoun
NG
,
Saini
V
,
Sessler
DI
: .
Validation and calibration of the Risk Stratification Index.
Anesthesiology
.
2017
;
126
:
623
30
44.
McIsaac
DI
,
Wijeysundera
DN
,
Huang
A
,
Bryson
GL
,
van Walraven
C
: .
Association of the hospital volume of frail surgical patients cared for with outcomes after elective, major noncardiac surgery: A retrospective population-based cohort study.
Anesthesiology
.
2017
;
126
:
602
13
45.
Lawson
EH
,
Louie
R
,
Zingmond
DS
,
Brook
RH
,
Hall
BL
,
Han
L
,
Rapp
M
,
Ko
CY
: .
A comparison of clinical registry versus administrative claims data for reporting of 30-day surgical complications.
Ann Surg
.
2012
;
256
:
973
81
46.
Charest-Morin
R
,
Street
J
,
Zhang
H
,
Roughead
T
,
Ailon
T
,
Boyd
M
,
Dvorak
M
,
Kwon
B
,
Paquette
S
,
Dea
N
,
Fisher
CG
,
Flexman
AM
: .
Frailty and sarcopenia do not predict adverse events in an elderly population undergoing non-complex primary elective surgery for degenerative conditions of the lumbar spine.
Spine J
.
2018
;
18
:
245
54
47.
Searle
SD
,
Mitnitski
A
,
Gahbauer
EA
,
Gill
TM
,
Rockwood
K
: .
A standard procedure for creating a frailty index.
BMC Geriatr
.
2008
;
8
:
24
Appendix 1.

American College of Cardiology and American Heart Association Guidelines Based Surgical Risk and Health Quest Score-based Stratification of Preoperative Patients for Preoperative Anesthesia Consultation and Evaluation Clinic Visit

American College of Cardiology and American Heart Association Guidelines Based Surgical Risk and Health Quest Score-based Stratification of Preoperative Patients for Preoperative Anesthesia Consultation and Evaluation Clinic Visit
American College of Cardiology and American Heart Association Guidelines Based Surgical Risk and Health Quest Score-based Stratification of Preoperative Patients for Preoperative Anesthesia Consultation and Evaluation Clinic Visit

Appendix 2. Rationale for Waiver of Written Consent, Approved by the Cleveland Clinic Institutional Review Board

  • a. The American College of Surgeons recommend obtaining frailty index (using Hopkins Frailty Score) preoperatively (http://site.acsnsqip.org/wp-content/uploads/2011/12/ACS-NSQIP-AGS-Geriatric-2012-Guidelines.pdf; accessed August 20, 2014) as a part of their best practice guidelines. It is recommended for better preoperative risk stratification and also for identification/documentation of a preoperative baseline.

  • b. Aside from frailty, no additional information was obtained that deviates from routine assessment. Patients were not be contacted subsequently because all other necessary data were available in our electronic medical records.

  • c. This study did not influence patient care or outcome.

Patient Information Sheet: The Cleveland Clinic Foundation Research Information Sheet

Study title: What Is the Best Measure of Frailty?

Investigator: Daniel I. Sessler, M.D.

The Cleveland Clinic Anesthesia Center is conducting a research study to determine the best way to measure frailty. Frailty is a measure of physical strength and weakness. Patients who are frail may be at higher risk of complications after surgery. We are interested in studying the best way to measure frailty before surgery.

You are being asked to participate in this study because you are an adult having noncardiac surgery.

Participation involves:

  • Completion of Hopkins Frailty questionnaire and medical record review.

  • The questionnaire consists of 3 questions and 2 assessments. You will walk for 15 feet for 3 trials. You will squeeze a grip device 3 times. In total, this will take about 10 min.

  • Completing these activities is all you need to do.

  • Review of your medical records after your surgery.

Additional information:

  • Risk is minimal. There is no physical risk. There is potential risk to your information. However, your data will be protected by maintaining it on a secure network drive on a password-protected computer. In addition, data will only be accessed by study staff.

  • There is no direct benefit to you. Knowledge gained will assist us to make an improved perioperative assessment tool.

  • Participation is voluntary. Your decision to participate or not will have no impact on current or future medical care at the Cleveland Clinic.

  • For questions related to the research study, please ask or contact Dr. Daniel Sessler at 216-444-4900.

  • For questions about your rights as a research subject, call the Institutional Review Board at 216-444-2924.

  • Completion of the questionnaire and activities will indicate your agreement to participate in the research study.