Recent reports have raised concerns that public reporting of 30-day mortality after cardiac surgery may delay decisions to withdraw life-sustaining therapies for some patients. The authors sought to examine whether timing of mortality after coronary artery bypass graft surgery significantly increases after day 30 in Massachusetts, a state that reports 30-day mortality. The authors used New York as a comparator state, which reports combined 30-day and all in-hospital mortality, irrespective of time since surgery.
The authors conducted a retrospective cohort study of patients who underwent coronary artery bypass graft surgery in hospitals in Massachusetts and New York between 2008 and 2013. The authors calculated the empiric daily hazard of in-hospital death without censoring on hospital discharge, and they used joinpoint regression to identify significant changes in the daily hazard over time.
In Massachusetts and New York, 24,864 and 63,323 patients underwent coronary artery bypass graft surgery, respectively. In-hospital mortality was low, with 524 deaths (2.1%) in Massachusetts and 1,398 (2.2%) in New York. Joinpoint regression did not identify a change in the daily hazard of in-hospital death at day 30 or 31 in either state; significant joinpoints were identified on day 10 (95% CI, 7 to 15) for Massachusetts and days 2 (95% CI, 2 to 3) and 12 (95% CI, 8 to 15) for New York.
: In Massachusetts, a state with a long history of publicly reporting cardiac surgery outcomes at day 30, the authors found no evidence of increased mortality occurring immediately after day 30 for patients who underwent coronary artery bypass graft surgery. These findings suggest that delays in withdrawal of life-sustaining therapy do not routinely occur as an unintended consequence of this type of public reporting.
Public reporting of patient outcomes may have unintended consequences on provider decision-making
It is unclear whether public reporting of 30-day coronary artery bypass graft mortality is associated with changes in end-of-life decisions near the 30-day reporting window
The authors compared patient outcomes in Massachusetts, a state with 30-day coronary artery bypass graft mortality reporting, and New York, a state with 30-day and all in-hospital mortality reporting
There were no meaningful differences in mortality rates near the 30-day reporting window in either state
This analysis suggests that a focus on reporting 30-day mortality was not associated with a difference in mortality near that reporting window cutoff
THE importance of measuring quality metrics is well accepted in medicine. However, when metrics are publicly reported, there are often concerns that there may be unintended consequences.1–6 Examples of such actions include the practice of upcoding, where hospitals may code a higher number of diagnoses to inflate patients’ severity of illness, leading to an improvement in the appearance of risk-adjusted outcomes or higher reimbursement.2,3 In patients with pneumonia, a condition for which 30-day mortality and readmissions are tracked by the Centers for Medicare and Medicaid Services (Baltimore, Maryland), there is the suggestion that some hospitals may reclassify patients into another diagnostic category without publicly reported outcomes.4–6
One quality metric that has received particular scrutiny is the measurement of 30-day mortality after surgical procedures. The 30-day mortality metric has been criticized for not being patient centered and for creating a “perverse incentive” for surgeons.7 Results from a national survey suggested that surgeons who reported being concerned with performance metrics were significantly less likely to offer surgery to patients who prefer to limit life-sustaining therapies.8 Also, due to a strong sense of responsibility for patient outcomes, surgeons already struggle with the decision to withdraw postoperative life-supporting therapies9 ; concerns for such metrics may further complicate the decision-making process.
Several reports have suggested that public reporting of 30-day mortality metrics in cardiac surgery may lead to delayed decisions to withdraw life-sustaining therapies for some patients to prevent their deaths from counting as adverse events.10–13 Investigating such unintended consequences of public reporting in cardiac surgery has become particularly salient, because the Centers for Medicare and Medicaid Services began posting 30-day metrics for patients undergoing coronary artery bypass graft (CABG) surgery in October 2015 (available at https://www.medicare.gov/hospitalcompare/search.html). However, public reporting after cardiac surgery is not universal, and different approaches to measuring mortality exist. One such metric is all-cause mortality measured at 30 days, where any deaths that occurred within 30 days of surgery are counted. Conceivably, this is a metric that may be affected by delays in the withdrawal of life-sustaining therapy, because deaths occurring during hospitalization after day 30 are not counted. However, another commonly used metric is the combination of 30-day mortality and all in-hospital mortality; because all in-hospital deaths are counted, this latter metric would not be affected by delays in the withdrawal of life-sustaining therapy.
If public reporting of surgical outcomes was to affect the timing of withdrawal of life-sustaining therapy, we hypothesized that this would be most likely to occur in a state using the 30-day mortality metric. Thus, we sought to examine whether this potential phenomenon exists on a large scale by measuring the timing of death for patients who had CABG surgery in Massachusetts, a state that publicly reports 30-day mortality rates. We used New York for comparison, a state that publicly reports the combined metric of 30-day and all in-hospital mortality.
Materials and Methods
The study protocol was reviewed and approved by the institutional review board of Columbia University Medical Center (New York, New York). Written informed consent was waived. Data for this study came from two state discharge databases, the Massachusetts State Inpatient Database and the New York Statewide Planning and Research Cooperative System, containing all hospitalizations within Massachusetts and New York for the years 2008 to 2013. For both states, the occurrence of patient deaths after hospital discharge for public reporting is identified by linkage to state vital records databases.
We chose to use these specific databases because both states have a long history of publicly reporting outcomes in cardiac surgery, with reporting of hospital outcomes dating back to 2002 in Massachusetts and 1990 in New York, and reporting of surgeon-specific outcomes starting in 2004 for Massachusetts and in 1992 for New York.14 We chose to use data from states that publicly reported outcomes, because surgeons in these states would be most likely to feel pressure and, thus, may be less likely to withdraw life-sustaining therapy. However, Massachusetts and New York differ in their cardiac surgery risk-adjusted mortality metrics (table 1). In Massachusetts, the metric consists solely of any mortality occurring within 30 days of surgery (including in-hospital deaths, as well as those occurring after hospital discharge),15 whereas New York uses a combined metric of any mortality within 30 days of surgery and all deaths occurring during the same hospitalization as the surgical procedure, irrespective of the time since surgery.16 Also, Massachusetts reports data only for CABG surgery, whereas New York reports outcomes for all cardiac surgeries. Thus, using data from Massachusetts would allow us to test the hypothesis that use of a 30-day mortality metric may be associated with an increase in in-hospital deaths occurring after day 30, with data from New York serving as a negative control.
We identified patients who had CABG surgery using Clinical Classifications Software Procedure codes.17 We summarized demographic and clinical characteristics for patients undergoing CABG in each state, including age, sex, race, insurance, number of Elixhauser comorbidities,18 use of mechanical ventilation for more than 96 h or dialysis during hospitalization, and hospital length of stay, and we compared characteristics for all patients between Massachusetts and New York. We also compared these characteristics within each state for patients who did and did not die in the hospital, using t tests, Mann–Whitney U, and chi-square tests, as appropriate.
We quantified the absolute number of in-hospital deaths for each postoperative day. A priori, we determined that the primary outcome would be the empirical daily hazard for in-hospital mortality. Using survival analysis and life tables, we calculated time from day of surgery to death and determined the empirical daily mortality hazard without censoring. We chose not to censor patients at time of hospital discharge, because the hazard becomes misleading given the high frequency of early discharges. This daily hazard rate represents the probability of mortality in the hospital, conditional upon survival to a given day, and was calculated up to 70 days after surgery; all patients were censored at day 70.
We used joinpoint regression to identify significant changes in the empirical daily mortality hazard over time.19 We chose to use this particular methodology because this would detect a significant increase in the empirical hazard occurring over time and the given day on which that increase begins (e.g., day 31) through the identification of a significant joinpoint on that day. Joinpoint regression analyzes time-ordered data to identify significant changes in an outcome occurring over time and has been widely used to study rates of change in the incidence of disease.20–22 The identified points of change, or joinpoints, separate the data into different segments, and the joinpoint model then tests whether differences in the rate of change in the outcome (i.e., the slope) surrounding the joinpoints are significant. In contrast to piecewise or segmented regression (which also tests differences in slopes around a selected time point), in joinpoint regression, the points of change are empirically determined from the data. For example, identification of significant joinpoints at days 31 and 42 would suggest that the slope of the daily hazard between days 31 and 42 is significantly different than the slope of the hazard before day 31 and after day 42. However, joinpoint regression will not detect an isolated single daily increase; for this, we visually inspected the data to identify possible increases and planned to test any differences between log daily hazards using t tests with bootstrapping to estimate 95% CIs for the ratio of the hazards.
For the joinpoint regression, we specified use of a log-linear model because the distribution of the daily hazard was nonnormal. We also specified a maximum number of five joinpoints and use of the permutation test for model selection, with 4,500 permutations for Monte Carlo simulation. We chose the permutation test because this method has been demonstrated to select the most parsimonious model with the best fit (https://surveillance.cancer.gov/joinpoint/faq/permutation_vs_bic.html). The permutation test repeatedly tests two different joinpoint models: a simpler model with fewer joinpoints (the null model) and a more complicated model (the alternative model). The test will determine whether the improved fit of the more complicated, alternative model is better than would be expected by chance; if so, then the null model is rejected and the alternative model is selected as the optimal model.
For other model parameters, we specified a minimum of three observations for the minimum number of observations from a joinpoint to either end of the data and a minimum of four observations for the minimum number of observations between two joinpoints (both are the default in the Joinpoint software). To account for the possibility of nonindependence between the daily hazards, we instructed the Joinpoint software to fit a model with autocorrelated errors based empirically on the data; however, using this method may decrease the power of the model to detect joinpoints. Consequently, we also fit a model with uncorrelated errors to ensure that our results were unchanged. Database management and statistical analysis were performed using Stata 13.1 (Stata Corp LP, USA) and Joinpoint 22.214.171.124 (NCI Surveillance Research Program, USA).
Between 2008 and 2013, 24,864 patients in Massachusetts and 63,323 patients in New York underwent CABG surgery. The overall rate of in-hospital mortality was low, with only 524 deaths occurring in Massachusetts (2.1%) and 1,398 occurring in New York (2.2%). In comparison with patients undergoing CABG in New York, patients in Massachusetts were more likely to be white (87% vs. 72% for New York; P < 0.001) and have fewer comorbidities (16% with four or more comorbidities vs. 32% for New York; P < 0.001). Consistent across both states, patients who died in hospital (in comparison with patients who survived), were more likely to be older (73 vs. 67 yr for Massachusetts, 73 vs. 68 yr for New York; P < 0.001), women (41% vs. 28% for Massachusetts, 41% vs. 25% for New York; P < 0.001), have more comorbidities (four or more comorbidities, 24% vs. 16% for Massachusetts, 33% vs. 32% for New York; P < 0.001), and be insured by Medicare (67% vs. 49% for Massachusetts, 76% vs. 57% for New York; P < 0.001). Patients dying in hospital were also more likely to receive mechanical ventilation for more than 96 h (35% vs. 3% for Massachusetts, 44% vs. 3% for New York; P < 0.001) and dialysis during hospitalization (27% vs. 2% for Massachusetts, 27% vs. 3% for New York; P < 0.001) and have longer hospital lengths of stay (14 vs. 8 days for Massachusetts, 11 vs. 8 days for New York; P < 0.001; table 2).
The number of in-hospital deaths occurring on days 31 to 37 was low in both states, with 18 deaths in Massachusetts (3.4% of all deaths) and 63 in New York (4.5% of all deaths) occurring over the 6-yr study period (fig. 1). In visually examining the daily hazards, there was no obvious increase in the empiric daily hazard for in-hospital death after day 30 in either state, with substantial noise in the data. Joinpoint regression revealed a significant change point in the daily hazard at day 10 (95% CI, 7 to 15) in Massachusetts and days 2 (95% CI, 2 to 3) and 12 (95% CI, 8 to 15) in New York but none at or after day 30 or 31 in either state (fig. 2; table 3).
In a state with a long-standing history of publicly reporting 30-day mortality after CABG, we found no evidence of an increase in in-hospital deaths occurring after the 30-day mark, suggesting that use of this quality metric does not result in obvious delays in altering goals of care for patients dying in the hospital. Furthermore, the number of in-hospital deaths occurring around and after day 30 was very low (less than 5% of all deaths), suggesting that even if this phenomenon exists, it is unlikely to have a major impact on the delivery of appropriate end-of-life care for most patients after CABG surgery.
Although there are several theoretical concerns about the negative unintended effects of public reporting in cardiac surgery, existing studies have largely examined upcoding (an increase in the coding of comorbidities) and risk aversion,23 where surgeons decline to operate on higher-risk patients.24–26 A previous study using a national data set to examine the timing of death after any cardiac surgery did find an increase in the daily hazard at day 30 but included data from states with and without public reporting and did not quantify this increase in terms of absolute numbers of deaths.10 Our findings may differ because we limited our cohort to patients who underwent a surgical procedure that is definitely subject to public reporting, with use of a metric that is potentially modifiable by delays in end-of-life decision-making. Moreover, our findings empirically support results of a national survey of surgeons, where concern about performance metrics was not associated with differences in the likelihood of withdrawing life-sustaining therapy9 and are similar to a study of patients undergoing a number of different surgical procedures that did not find an increase in mortality occurring after 30 days.27
Anecdotally, concerns about the impact of 30-day mortality metrics on surgeons’ decision-making are widespread.28 It is important to note that surgeons may have a general reluctance to move toward withdrawal of life support. The culture of “never giving up” has been well documented,13,29 and surgeons may be perceived as providing suboptimal care or influencing outcomes for nonpatient-centered reasons, leading to frequent conflict around postoperative goals of care.30 However, there may be a misattribution of the role of 30-day mortality metrics as a motivating factor for surgeons’ reticence to alter goals of care; the perceived reticence to move toward providing appropriate end-of-life care is more likely due to the strength of the contractual patient–surgeon relationship that occurs preoperatively and a surgeon’s sense of personal responsibility for outcomes.9,29 Dispelling such negative concerns about surgeons’ motivations in the postoperative period may help to decrease conflict between surgeons and the other providers who care for these patients.
Given differences in both national and statewide initiatives regarding the procedures and metrics reported (as illustrated in table 1), clinicians may not always know which metrics are relevant in their practice setting. A case report from Massachusetts that drew national attention to the use of 30-day mortality metrics involved an older woman who underwent transaortic valve repair (a procedure for which 30-day mortality is not publicly reported) for whom a goals-of-care discussion was deferred until after day 30.11,28 Although this case is striking and points to the possibility that clinician behaviors may not necessarily be tied to applicable metrics, our findings indicate that such occurrences are more likely to be an exception than the norm.
Although the use of 30-day mortality metrics in surgery has been recently criticized,7,28 of the various programs that participate in public reporting, the majority of statewide initiatives, as well as the Society for Thoracic Surgeons database, measure a combination of 30-day mortality and all in-hospital mortality, regardless of timing. Although the combined metric would be unaffected by delays in decisions to proceed with withdrawal of life-sustaining therapies, it may be affected by other actions, such as earlier discharge to long-term acute care or skilled nursing facilities. Because cardiac surgery patients comprise approximately 15% of chronic critically ill Medicare patients who are discharged to long-term acute care hospitals,31 and studies suggest that this type of “discharge bias” can affect hospital rankings of quality,32–34 the possibility of these other unintended consequences should also be investigated.
Our study was limited by the fact that we were unable to identify and describe discussions about end-of-life care or determine the timing of deaths after decisions to withdraw life-sustaining therapies. We also could not determine whether the postoperative care provided was aligned with patients’ and families’ preferences and whether care was provided solely by the surgeon who operated or by other clinicians, such as anesthesiologists, intensivists, and palliative care physicians. It is possible that we were underpowered to detect a difference in the rate of late postoperative deaths. However, we actually observed a decrease in mortality occurring on postoperative days 31 to 37 (in comparison with days 24 to 30). Furthermore, given the low number of deaths occurring after postoperative day 30 in either state over the 6-yr study period, any phenomenon affecting these late deaths is unlikely to have a large clinical impact. Our study also focuses solely on patients undergoing CABG, who have a relatively low risk of in-hospital death compared to patients undergoing many other surgical procedures; it is possible that practices may vary depending on the type of surgery assessed. Finally, we only examined care in two states in the United States, and our findings may not be generalizable to other locations.
With the national push to increase transparency and improve the quality of care through public reporting, we address an important issue regarding the potential unintended consequences of instituting mortality metrics. Although these findings refute the concern that public reporting of mortality may lead to widespread inappropriate end-of-life decision-making by physicians, strict mortality metrics also may not be the best patient-centered measure of surgical quality.7,35 Focusing on surgical patient selection and decision-making before surgery may yield greater benefits for improving the quality of care for these patients.13,29
Data for this article were obtained from the New York Statewide Planning and Research Cooperative System (Albany, New York), New York State Department of Health (Albany, New York), and the New York Department of Vital Statistics, New York State Department of Health (Albany, New York). The information contained herein was derived from data provided in part by the Bureau of Vital Statistics, New York City Department of Health and Mental Hygiene (New York, New York).
Supported by Paul B. Beeson Career Development Awards (award No. K08AG051184 to Dr. Hua; award No. K76AG054859 to Dr. Cooper) from the National Institute on Aging, National Institutes of Health (Bethesda, Maryland) and the American Federation for Aging Research (New York, New York); a Fellowship in Translational Health Research from the Physicians’ Services Incorporated Foundation (Toronto, Ontario, Canada; to Dr. Scales); and the Cambia Foundation Sojourns Scholarship (Portland, Oregon; to Dr. Cooper).
The authors declare no competing interests.