Abstract
Frailty is not uncommon in the surgical population, with reported incidences of 2 to 45%, depending on the assessment used and the population assessed
Frailty has been associated with adverse outcomes after a variety of surgical procedures, elective or emergent, and in different age groups
Frailty can be assessed with various approaches and measurement tools
The Hopkins Frailty Score (a phenotype model) and the Modified Frailty Index score (a measure of deficit accumulation) were both poor predictors of unexpected prolonged hospital stay and a composite of readmission and serious complications
Frailty is associated with adverse postoperative outcomes, but it remains unclear which measure of frailty is best. This study compared two approaches: the Modified Frailty Index, which is a deficit accumulation model (number of accumulated deficits), and the Hopkins Frailty Score, which is a phenotype model (consisting of shrinking, weakness, exhaustion, slowness, and low physical activity). The primary aim was to compare the ability of each frailty score to predict prolonged hospitalization. Secondarily, the ability of each score to predict 30-day readmission and/or postoperative complications was compared.
This study prospectively enrolled adults presenting for preanesthesia evaluation before elective noncardiac surgery. The Hopkins Frailty Score and Modified Frailty Index were both determined. The ability of each frailty score to predict the primary outcome (prolonged hospitalization) was compared using a ratio of root-mean-square prediction errors from linear regression models. The ability of each score to predict the secondary outcome (readmission and complications) was compared using ratio of root-mean-square prediction errors from logistic regression models.
The study included 1,042 patients. The frailty rates were 23% (Modified Frailty Index of 4 or higher) and 18% (Hopkins Frailty Score of 3 or higher). In total, 12.9% patients were readmitted or had postoperative complications. The error of the Modified Frailty Index and Hopkins Frailty Score in predicting the primary outcome was 2.5 (95% CI, 2.2, 2.9) and 2.6 (95% CI, 2.2, 3.0) days, respectively, and their ratio was 1.0 (95% CI, 1.0, 1.0), indicating similarly poor prediction. Similarly, the error of respective frailty scores in predicting the probability of secondary outcome was high, specifically 0.3 (95% CI, 0.3, 0.4) and 0.3 (95% CI, 0.3, 0.4), and their ratio was 1.00 (95% CI, 1.0, 1.0).
The Modified Frailty Index and Hopkins Frailty Score were similarly poor predictors of perioperative risk. Further studies, with different frailty screening tools, are needed to identify the best method to measure perioperative frailty.
Frailty denotes an increased vulnerability or a diminished capacity to cope with external stressors. The pathophysiology of this poorly characterized and complex clinical syndrome remains poorly understood, but it is often attributed to reduced physiologic reserve. Frailty has been associated with adverse outcomes after a variety of surgical procedures, elective or emergent, and in different age groups.1–4 The reported incidence of frailty among patients presenting for noncardiac surgery ranges from 2 to 13%, depending on the frailty assessment tool used and definition thresholds.5,6 The incidence of frailty is substantial in high-risk populations, including patients having emergency surgery (26%)7 and elderly patients having major orthopedic surgery (41%).8
Initially, frailty measurement tools were constructed to predict mortality and other adverse events such as falls, disability, and hospitalization among community-dwelling elders.9–12 Such assessment tools have been extended to characterize frailty in the perioperative setting.4 However, the best method to assess perioperative frailty remains unknown,13 perhaps explaining why frailty is not routinely used for perioperative risk stratification.14
There are two broad approaches to characterizing frailty.15 The first, and best studied, is the deficit accumulation model. This approach assumes that frailty occurs due to accumulation of multiple deficits across various domains.11 The Modified Frailty Index is a deficit accumulation model based on the proportion of comorbidities present in a patient.16,17 It is typically calculated with data from the patient’s medical record or from a perioperative clinical database. The second approach is the phenotype model, which defines frailty as a clinical syndrome, determined by quantitatively measuring five clinical features: shrinking, weakness, exhaustion, slowness, and low physical-activity level.12 These variables are rarely recorded during preoperative assessments and are thus not readily available in electronic records. A perioperative example is the Hopkins Frailty Score.18
Both the Modified Frailty Index and the Hopkins Frailty Score predict adverse outcomes after surgery.1,5,18–22 It is unclear, however, which paradigm of frailty assessment better assesses perioperative risk. Because there is no gold standard to validate these frailty assessment tools, comparison of their abilities to predict adverse postoperative outcomes would be most useful. Thus, the primary aim of this study was to compare the Modified Frailty Index and the Hopkins Frailty Score on their ability to predict prolonged hospitalization, measured as the difference between actual and expected surgery-specific postoperative length of stay. Secondarily, we compared these two tools on their ability to predict the secondary outcome, a composite of 30-day readmission and various serious postoperative complications.
Materials and Methods
Adults presenting to the Preoperative Anesthesia Consultation and Evaluation Clinic at the Cleveland Clinic (Cleveland, Ohio) Main Campus were considered for inclusion into this prospective observational cohort study. Approval for this study was obtained from the Cleveland Clinic Institutional Review Board.
Patients scheduled for noncardiac surgery at our institution are initially triaged using a model adapted from the American College of Cardiology (Washington, D.C.) and American Heart Association (Dallas, Texas) guidelines based classification of surgical risk and an online health survey score, HealthQuest, developed by the Department of General Anesthesiology, Cleveland Clinic, Cleveland, Ohio.23 Using this process, about half of our surgical patients are triaged to be evaluated at the preoperative evaluation clinic (appendix 1).
Patients at the preoperative clinic are arbitrarily assigned to evaluation by registered nurses trained in preoperative evaluation or anesthesiology residents, both supervised by an attending anesthesiologist. Frailty assessments are not routinely conducted. Three of the preoperative nurses were trained to administer a study-specific questionnaire and collect data for calculation of the Hopkins Frailty Score. All patients presenting to these specific trained providers were included in the study. Because frailty assessment is a recommended part of preoperative assessment for patients, based on American College of Surgeons (Chicago, Illinois) best practice guidelines, written informed consent was waived by the Institutional Review Board. However, patients were given written information about the study and had the option not to participate (appendix 2). Verbal consent was obtained by the nurse who did the assessment.
Measurements
The Modified Frailty Index includes 11 variables (table 1). The occurrence of each variable, except New York Heart Association (New York, New York) functional classification, was determined based on International Classification of Diseases (9th or 10th revision)–based billing codes for each variable associated with the surgical hospitalization. These data were retrieved from the Cleveland Clinic Perioperative Health Documentation System registry once all patients were recruited. This registry captures perioperative data for all patients undergoing surgery at Cleveland Clinic. The functional classification was assessed by the interviewer during the preoperative visit and documented in our REDCap study database.
The Hopkins Frailty Score includes five components, specifically shrinking, exhaustion, low physical activity, slowed walking speed, and weakness (fig. 1).12,18 Each component is given a score of 1 when present, and the cumulative score forms the Hopkins Frailty Score for each patient. Physical activity was measured based on the short version of the Minnesota Leisure Time Activity questionnaire.24 Grip strength was measured using the dominant hand with a hand-held dynamometer (Sammons Preston Rolyan, Jamar Technologies, USA). Weakness was defined as a reduction in grip strength based on sex and body mass index–based normative data. Similarly, slow walking speed was determined based on normative data for sex and height. An average of three consecutive measurements was used to determine the presence of slowed walking speed and a reduction in grip strength. Normative data–based cutoffs for walking speed and grip strength used in the Hopkins Frailty Score are reported in figure 1.
Assessment criteria for calculating Hopkins Frailty Score. Reprinted from Journal of American College of Surgeons, 210(6), Makary MA, Segev DL, Pronovost PJ, Syin D, Bandeen-Roche K, Patel P, Takenaga R, Devgan L, Holzmueller CG, Tian J, Fried LP, Frailty as a predictor of surgical outcomes in older patients, pages 901-8, Copyright (2010), with permission from Elsevier.
Assessment criteria for calculating Hopkins Frailty Score. Reprinted from Journal of American College of Surgeons, 210(6), Makary MA, Segev DL, Pronovost PJ, Syin D, Bandeen-Roche K, Patel P, Takenaga R, Devgan L, Holzmueller CG, Tian J, Fried LP, Frailty as a predictor of surgical outcomes in older patients, pages 901-8, Copyright (2010), with permission from Elsevier.
The Hopkins Frailty Score assessments were entered into an iPad (Apple, Inc., USA) during the preoperative visit and transmitted to our online REDCap study database. They were thus not incorporated into the electronic medical record. Patients with a Hopkins Frailty Score of 3 or more are classified as frail.12,25 Among various cutoffs used to designate frailty based on the Modified Frailty Index, the score of 4 or higher out of 11 is most commonly reported.6,26 However, both scores were considered as continuous variables for our analyses.
The primary and secondary outcomes were calculated based on data retrieved from the Cleveland Clinic Perioperative Health Documentation System registry. Length of stay for our study was defined as the total number of postoperative days spent in the hospital. When patients were released from the hospital on the day of surgery, we assigned patient length of stay to be 1 day. Length of stay is influenced by two relevant non–patient-related factors: the type of surgery and hospital-related factors. To adjust for these confounders, our primary outcome—prolonged hospitalization—was defined as the difference between the actual and expected length of hospitalization (prolonged hospitalization = actual length of stay – expected length of stay). The actual length of stay was the true postoperative length of hospitalization for each patient in our cohort. The expected length of stay (hospital- and surgery-specific) for a patient was estimated based on the median length of postoperative hospitalization for all adults between January 1, 2010 and December 31, 2015 who had same procedure according to the Current Procedural Terminology procedure code at our institution.
The secondary outcome was a binary composite that included 30-day complications and 30-day readmission after the noncardiac surgery. Postoperative complications were defined a priori based on the postoperative morbidity survey described by Bennett-Guerrero et al.27 and the Agency for Healthcare Research and Quality reported postoperative complications (table 2) and were retrieved from the Cleveland Clinic Perioperative Health Documentation System registry using International Classification of Diseases (9th or 10th revision)–based codes for surgical admission or readmission within 30 days. Because all postoperative complications that might occur in frail patients are unknown, we included readmission as an outcome because it captures various serious events. The occurrence of 30-day readmission was captured from the Cleveland Clinic Perioperative Health Documentation System registry, which included readmission to any Cleveland Clinic Health System hospital.
Statistical Analysis
Primary Analysis.
For the first simplified description of the relationship between the Modified Frailty Index and Hopkins Frailty Score, we used the Pearson correlation coefficient; Fisher’s z transformation was used to report confidence limits on the correlation coefficient. The analysis described below addressed the primary aim of the study comprehensively.
For the primary analysis, the accuracy with which each frailty score predicts prolonged hospitalization was calculated using root-mean-squared prediction error (prediction error); that is, the average distance between the observed and predicted outcome. If a frailty score improves the accuracy with which we predict prolonged hospitalization, model residuals are smaller, thereby reducing the root-mean-squared prediction error. Controlling for bias imposed by a given sample, bootstrap resampling28 cross-validation techniques were used to compare the prediction errors between two linear regression models with prolonged hospitalization as the outcome and the two frailty indices as predictors.
Based on the original data set, we generated a bootstrap sample using sampling with replacement, randomly partitioning our sample into five equal subsamples and calculating five-fold cross-validated prediction errors for the Modified Frailty Index and Hopkins Frailty Score regression models. The ratios of root-mean-squared prediction error between the Modified Frailty Index and Hopkins Frailty Score were calculated as well. We repeated bootstrap sampling 10,000 times. Cross-validated prediction errors were averaged over the 10,000 bootstrap data sets for both the Modified Frailty Index and Hopkins Frailty Score models, with the lowest value indicating the best predictive model; 95% bootstrap confidence intervals were also reported.
To evaluate the primary aim of the study, which is to compare the abilities of the Modified Frailty Index and Hopkins Frailty Score to predict prolonged hospitalization, a 95% bootstrap confidence interval of prediction error ratio was constructed. If the confidence interval for this ratio did not include one, then a statistically significant difference in predictive ability could be claimed.
Two final linear regression models for the Modified Frailty Index and Hopkins Frailty Score scoring systems were fit using the entire data set. The model based two-sided Wald test P values were reported to measure univariate association between the frailty indices and the primary outcome (prolonged hospitalization), with significance criteria set at P < 0.05. Discrimination ability was estimated via coefficient of determination R2 (the proportion of the variance in outcome explained by the model; R2 indicates how well data fit a statistical model for the continuous outcome) for each model. Calibration of the two models was assessed by plotting predicted versus observed outcome (prolongation in hospitalization) averaged over patients with identical predicted values. Ideal calibration would be indicated by values close to the 45° line on a plot.
As a sensitivity analysis, we performed the primary analysis using log-transformed actual length of hospitalization as the primary outcome instead of prolonged hospitalization. Log transformation made the data more normally distributed.
Secondary Analysis.
To evaluate the secondary aim of the study, which is to compare the ability of the Modified Frailty Index and Hopkins Frailty Score to predict a composite outcome of 30-day complications and readmission, we used algorithms similar to those used for the primary analysis. Because the secondary outcome is a binary outcome, the two compared models were logistic regression models. R statistical software version 2.15.1 for 64-bit Microsoft Windows (R Foundation for Statistical Computing, Austria) and SAS software version 9.4 (SAS Institute, USA) were used for statistical analyses.
Sample Size and Power Analysis.
The sample size calculation was based on the primary aim of comparing the Modified Frailty Index and Hopkins Frailty Score models on their ability to predict prolongation in postoperative length of hospitalization after noncardiac surgery. For the purpose of sample size estimation, we used an F-test for comparing coefficients of determination R2 for two nested regression models. The significance level was set at 0.05 for the primary aim.
Hypothesizing that the R2 of the two models (coefficients of determination) would be 0.30 and 0.33 and assuming a standard deviation of 0.15, 1,000 patients would provide approximately 94% power for a two-sided test comparing the discrimination ability of the two models. Adjusting for three interim analyses at each 25% of the planned enrollment, a maximum of 1,190 patients were required.
Results
Between March 2, 2015 to November 9, 2016, 1,182 adult patients presenting to the preoperative clinic at the Cleveland Clinic main campus were included in this study. The study was stopped when we had 1,226 surveys and believed 1,190 unique patients were enrolled; however, 38 surveys were incomplete, and 6 turned out to be duplicate patients.
We excluded patients with missing medical record numbers, patients who did not have surgery within 90 days of the preoperative visit, and patients with missing preoperative study assessments. We also excluded patients who had surgeries lasting less than 30 min. Finally, patients with incomplete outcome data for analysis were excluded. That left 1,042 unique patients in our analysis cohort (fig. 2).
The demographic data, baseline characteristics, types of surgeries, and frailty measures are reported in table 3. The distribution of Hopkins Frailty Score and Modified Frailty Index values across the population is shown in figure 3. Based on the Hopkins Frailty Score, 184 (18%) patients were classified as frail. Based on a Modified Frailty Index cutoff of 4 or greater, 234 (23%) patients of our cohort were classified as frail. The median length of stay was 2 days (interquartile range: 1, 4), and the median prolongation in hospitalization (primary outcome) was −1 day (interquartile range: −2, 0; table 4). The secondary outcome occurred in 12.9% of the patients. The most commonly reported postoperative complication was renal injury, occurring in 7.0% of the patients, and 6.5% of the patients were readmitted within 30 days of the surgery (table 4). The Pearson correlation (95% Fisher’s z transformation confidence limits) between the Modified Frailty Index and Hopkins Frailty Score scores was 0.28 (95% CI, 0.22 to 0.33), indicating a weak positive linear relationship between the two frailty indices.
The Demographic and Baseline Characteristics, including Frailty Indices for the Study Population (N = 1,042)

Summary of the Postoperative Outcomes Given as Medians [First Quartile, Third Quartile] or N (%), as Appropriate (N = 1,042)
![Summary of the Postoperative Outcomes Given as Medians [First Quartile, Third Quartile] or N (%), as Appropriate (N = 1,042)](https://asa2.silverchair-cdn.com/asa2/content_public/journal/anesthesiology/132/1/10.1097_aln.0000000000002959/3/m_13tt04.jpeg?Expires=1703004595&Signature=CG7n27XQp8JVipNnyJE5DoFYy2iW9gSTzrY3Rmpaaz-hdkVBqttFJCwmmP6sn7cR4mbX8aSmPfOOYweNltmvc6LRl3bXD89cHNA1nKfGAcY8uveYmt4F0sLhLOSv4p5z44jGrEN4Hd1Xj5sBntE9tN-RraVbhDwGvXPCUb4aGri~GEfOGJJln~kaEHfrUtN-WxUfk0SNHkANiK0aIEGV~bxkpYDjYZVhd-6~E754bsN9sCkLKMw4aNvj~wOYt6Bi9lrqagOgUSemY2l2phbI60a5Xth1xKgjA-o271ixuEnaZUTSGoHN02~~QyF1sYBNpJrduNhMOi~GnbebnTlFgQ__&Key-Pair-Id=APKAIE5G5CRDK6RD3PGA)
Distribution of Modified Frailty Index (MFI, A) and Hopkins Frailty Score (HFS, B) scores across the study cohort (N = 1,042).
Distribution of Modified Frailty Index (MFI, A) and Hopkins Frailty Score (HFS, B) scores across the study cohort (N = 1,042).
Primary Analysis
The prediction error of the Modified Frailty Index model was smaller than the Hopkins Frailty Score model, their ratio being 1.01 (95% CI, 1.00 to 1.02), indicating a statistically significant but clinically unimportant difference between the frailty indices in prediction accuracy (table 5). Despite the Modified Frailty Index being a significantly better predictor than the Hopkins Frailty Score, the prediction error for prolonged hospitalization was high (more than 2 days) for both scores. Thus, the ability of either frailty model to predict prolonged duration of hospitalization was poor. Similarly, the discrimination ability estimated via coefficient of determination (R2) was poor for both the Hopkins Frailty Score (0.002) and the Modified Frailty Index (0.023). Both calibration plots deviated substantially from the ideal 45° line for observed versus predicted prolongation in hospitalization (fig. 4).
Summary of the Prediction Models’ Performance for Difference between Actual and Expected Hospital Length of Stay and Probability of Postoperative Complication/Readmission Composite (N = 1,042)

Calibration plots of Modified Frailty Index (MFI, A) and Hopkins Frailty Score (HFS, B) models (N = 1,042).
Calibration plots of Modified Frailty Index (MFI, A) and Hopkins Frailty Score (HFS, B) models (N = 1,042).
The sensitivity analysis produced similar results. There was no difference between the Modified Frailty Index and Hopkins Frailty Score frailty assessments in prediction of log-transformed actual length of hospitalization. The bias corrected prediction error was 2.2 (95% CI, 2.1 to 2.2) days for both models (more than 2 days), and the estimated coefficient of determination R2 was 0.009 and 0.019, respectively.
Secondary Analysis
Table 5 also shows that the Modified Frailty Index and Hopkins Frailty Score were comparably bad at predicting the composite of 30-day readmission and postoperative complications, with the ratio of their prediction errors being 1.00 (95% CI, 0.99 to 1.01). Furthermore, the C statistic was 0.59 (95% CI, 0.54 to 0.64) for the Modified Frailty Index and 0.60 (95% CI, 0.55 to 0.65) for the Hopkins Frailty Score. The two measures thus predicted complications poorly and comparably so.
Discussion
In patients presenting for elective noncardiac surgery, the Modified Frailty Index better predicted differences between actual and expected hospital duration than the Hopkins Frailty Score, but not by a clinically important amount. Both scores were comparably bad at predicting a composite of 30-day postoperative complications and/or readmission. Thus, neither was a useful predictor of hospital duration or 30-day postoperative complications and/or readmission.
In a similar study, Cooper et al.8 evaluated 415 orthopedic surgery patients using the two paradigms of frailty. They used the Fried Index to measure the phenotype paradigm, which is similar to the Hopkins Frailty Score, but used a more comprehensive 40-point scale to assess the deficit accumulation paradigm. As in our study, Cooper et al.8 found that correlation between two paradigms of frailty was marginal. Neither was a good predictor of readmissions or postoperative complications, although frailty was associated with postoperative hospitalization exceeding 5 days (relative risk, 3.1). Other studies evaluating noncardiac surgical patients compared different deficit accumulation measurement tools but did not show clinical superiority of any one tool over the other.6,29,30
Our most surprising result, and therefore the most interesting, is that neither measure of frailty was a helpful predictor of length of hospitalization or 30-day complications and/or readmission. Both frailty scores resulted in errors exceeding 2 days in predicting surgery-specific length of hospitalization. Similarly, the C statistics for 30-day readmission and/or postoperative complications with our two assessment strategies were only 0.59 and 0.60 (with 0.5 indicating predictions no better than chance). Although statistically significant, these low values indicate that predictive ability was too weak to be clinically useful.
In contrast to our results, other investigations report that frailty, quantified with either deficit accumulation or phenotype models, is associated with prolonged hospital stay.1,8,20,31–33 Two recent meta-analyses34,35 report statistically significant odds ratios in a majority of the studies evaluating the impact of frailty on length of stay. However, substantial interstudy heterogeneity prevented pooling of study level effect estimates for length of stay. These results need to be interpreted with caution, because most studies use length-of-stay calculations that were not surgery-specific.1,8,31 For example, Flexman et al.1 used the National Surgical Quality Improvement Program database to evaluate 53,080 patients who had surgery for degenerative spine diseases. They found that the Modified Frailty Index was associated with prolonged hospitalization (odds ratio 1.88 for every 10% increase in the Modified Frailty Index; 95% CI, 1.81 to 1.99). However, their patients with the highest Modified Frailty Index scores had the most complex multilevel spine surgery. Hence, adjusting for the type of surgery as a confounder markedly decreased their odds ratio to 1.27.
Our study cohort was heterogeneous with patients having various noncardiac surgeries, each with its own expected length of postoperative hospitalization. We accounted for type of surgery as a confounder by separately estimating the expected duration of hospitalization for each type of surgery, based on the median length of stay for that procedure at our institution. Our approach improves on previous investigations that did not account for the type of surgery and used semiarbitrarily defined prolonged hospitalization. For instance, Partridge et al.33 defined prolonged hospitalization as a postoperative stay of more than 12 days after arterial vascular surgery. In addition, in contrast to others who reported the strength of association between frailty and length of stay using odds ratios,1,8,20,31–33 we measured the prediction error using root-mean-squared prediction error, which better quantifies clinical relevance. Our more robust analysis shows that there is little relationship between frailty and the prolongation in expected duration of hospitalization after noncardiac surgery, with an error exceeding 2 days. Our sensitivity analysis, evaluating the relationship between frailty and absolute length of hospitalization without accounting for type of surgery, also revealed a prediction error greater than 2 days for both indices.
Many investigators report significant associations between measures of frailty and postoperative complications,1,5,6,8,16,19,20,29,36–40 but only some report C statistics,6,29,30,33,36,37,39 which quantify clinical importance. Reported C statistics range from 0.53 to 0.68, with most being between 0.60 and 0.65,6,29,30,39 which are similar to our findings. A statistically significant odds ratio34,35 does not imply predictive ability or discriminative classification, making measures of discrimination such as the C statistic critical.41 The available evidence therefore suggests only weak associations between frailty and postoperative complications, presumably because many other factors contribute to postoperative morbidity.
Because frailty assessments are rarely included in administrative or clinical registries, frailty represents a potential unknown confounding factor. The fact that both deficit accumulation and phenotype paradigms of frailty are poor predictors of hospital duration and complications is therefore reassuring from a research perspective. Furthermore, various risk evaluation schemes effectively include components measured in the deficit accumulation paradigm. For example, the American College of Surgeons National Surgical Quality Improvement Program Risk Calculators is a surgery-specific risk assessment tool that shares many patient variables with the Modified Frailty Index.42 Similarly, the Risk Stratification Index models, objectively derived by identifying various International Classification of Diseases (9th revision)–based billing codes that predict adverse outcomes, include many that might be considered indicators of frailty.43 Therefore, omitting deficit accumulation or phenotype–based measurements of frailty may not introduce unobserved confounding in otherwise well adjusted registry analyses.
The incidence of frailty in our cohort is comparable with6,38 or higher than1,5,26 other large studies evaluating perioperative frailty. By virtue of recruiting patients triaged to preoperative clinic evaluation, we selected a higher-risk population, potentially increasing power. However, it remains possible that these frailty indices better discriminate postoperative adverse outcomes in a different surgical cohort than ours. Frail patients have better postoperative outcomes when they undergo surgery at centers with a high volume of frail patients.44 The Cleveland Clinic Main Campus is a high-volume center, and many of our patients are frail; for example, about 20% were frail in our current sample, and 80% were American Society of Anesthesiologists physical status III or IV. It seems likely that institutional experience with frail and high-risk patients in general mitigated the impact of frailty on postoperative outcomes.
The difference between the findings of our study and previous literature can be attributed, at least in part, to our robust study design, which analyzes prediction errors and discriminative ability of the models. Prior studies evaluating the relationship between frailty and postoperative outcomes commonly report statistically significant odds ratio which overlook the score’s clinical utility.
A limitation of our study is that the elements of the Modified Frailty Index and postoperative outcomes were obtained from an institutional registry and thus could be influenced by factors affecting capture of variables in administrative databases.45 Moreover, 30-day readmission might be underestimated because 15% of study patients were from out-of-state. However, missing readmissions will not change the validity of our conclusions because the comparisons were within patients. Although formal assessments of frailty obtained for our study were not included in the electronic medical record, clinicians could of course make their own assessments of frailty and modify perioperative management in response, thereby affecting outcome.
Because there is no gold standard for measuring frailty in the perioperative context, it is not possible to evaluate the accuracy of the Modified Frailty Index and Hopkins Frailty Score. Thus, predictive validation is an important aspect in all perioperative frailty studies. Many investigators report statistically significant associations between both of these frailty scores and adverse postoperative outcomes, thereby erroneously reporting good predictive potential in clinical practice.16,17 However, poor discriminative ability, as described in our study, makes them poor clinical tools for identifying patients at risk. Poor reliability or reproducibility of these tests may also be an important source of error in our study. Interrater reliability has not been reported for either the Modified Frailty Index or the Hopkins Frailty Score. Having a pretrained group of test givers, as in our study, presumably improves interrater reliability of a test. Moreover, objective tests such as the ones we used usually have high interrater reliability, although we did not qualify this aspect of reliability.
We tested only one measurement system from each frailty assessment paradigm, and it remains possible that other methods outperform the two we used. The ones we selected, however, are well validated in nonoperative contexts and are easy to administer perioperatively. The Modified Frailty Index, for example, is the most widely studied deficit accumulation model, and it predicts adverse postoperative outcomes in large administrative databases5,16,17 and small prospective studies.20,46 One may argue that the Modified Frailty Index consists of only 11 variables, which do not cover all the described domains of frailty. However, a total of 11 elements is similar to the 10-variable threshold suggested for developing a stable frailty assessment model.47 The Modified Frailty Index is also comparable to other deficit accumulation models including the risk analysis index and Ganapathi Index for predicting postoperative outcomes.6,36 Additionally, the Fried Index, which is the basis for the Hopkins Frailty Score, has been validated in a perioperative setting.18,21 Recently, Kapoor et al.39 compared the Fried Index with the self-reported Late Life Function and Disability Instrument and found them to be comparable in perioperative settings. It thus seems unlikely that our results would much differ had we selected other measures of frailty.
The poor predictive ability of Hopkins Frailty Score and Modified Frailty Index may reflect the fact that neither was developed to assess surgical risk. In fact, the deficit-accumulation model of frailty was initially developed using variables from the Canadian Study of Health and Aging as a proxy measure for aging, mortality, and other adverse events in community-dwelling elderly.10,11 Furthermore, neither the Hopkins Frailty Score nor the Modified Frailty Index has been directly validated against their comprehensive original models. Novel frailty assessment tools developed specifically for surgical risk assessment might provide better predictive accuracy.
In summary, the Hopkins Frailty Score and Modified Frailty Index were comparably poor predictors of prolonged hospitalization and a composite of readmission and serious complications. Despite previous reports of frailty being a useful marker of perioperative risk, neither of two well established measures was predictive in our patients. As a corollary, omitting measures of frailty seems unlikely to introduce substantial unobserved confounding in otherwise well adjusted registry analyses. Because ours was a single-center study evaluating two specific frailty assessment tools, further studies, especially with different frailty screening tools, are needed to identify the best method to measure perioperative frailty and to establish its clinical utility.
Research Support
Support was provided solely from institutional and/or departmental sources.
Competing Interests
The authors declare no competing interests.
References
Appendix 2. Rationale for Waiver of Written Consent, Approved by the Cleveland Clinic Institutional Review Board
a. The American College of Surgeons recommend obtaining frailty index (using Hopkins Frailty Score) preoperatively (http://site.acsnsqip.org/wp-content/uploads/2011/12/ACS-NSQIP-AGS-Geriatric-2012-Guidelines.pdf; accessed August 20, 2014) as a part of their best practice guidelines. It is recommended for better preoperative risk stratification and also for identification/documentation of a preoperative baseline.
b. Aside from frailty, no additional information was obtained that deviates from routine assessment. Patients were not be contacted subsequently because all other necessary data were available in our electronic medical records.
c. This study did not influence patient care or outcome.
Patient Information Sheet: The Cleveland Clinic Foundation Research Information Sheet
Study title: What Is the Best Measure of Frailty?
Investigator: Daniel I. Sessler, M.D.
The Cleveland Clinic Anesthesia Center is conducting a research study to determine the best way to measure frailty. Frailty is a measure of physical strength and weakness. Patients who are frail may be at higher risk of complications after surgery. We are interested in studying the best way to measure frailty before surgery.
You are being asked to participate in this study because you are an adult having noncardiac surgery.
Participation involves:
Completion of Hopkins Frailty questionnaire and medical record review.
The questionnaire consists of 3 questions and 2 assessments. You will walk for 15 feet for 3 trials. You will squeeze a grip device 3 times. In total, this will take about 10 min.
Completing these activities is all you need to do.
Review of your medical records after your surgery.
Additional information:
Risk is minimal. There is no physical risk. There is potential risk to your information. However, your data will be protected by maintaining it on a secure network drive on a password-protected computer. In addition, data will only be accessed by study staff.
There is no direct benefit to you. Knowledge gained will assist us to make an improved perioperative assessment tool.
Participation is voluntary. Your decision to participate or not will have no impact on current or future medical care at the Cleveland Clinic.
For questions related to the research study, please ask or contact Dr. Daniel Sessler at 216-444-4900.
For questions about your rights as a research subject, call the Institutional Review Board at 216-444-2924.
Completion of the questionnaire and activities will indicate your agreement to participate in the research study.