Abstract
Sugammadex provides rapid and effective reversal of neuromuscular blocks but is expensive
The investigators tested the hypothesis that a cognitive aid to guide selective use of sugammadex reduced use
They conducted a segmented regression (interrupted time series) retrospective analysis before and after implementing the cognitive aid and informational meetings for their department
Sugammadex use and associated costs, which were increasing, decreased substantially after introduction of the cognitive aid
The authors observed increased pharmaceutical costs after the introduction of sugammadex in our institution. After a request to decrease sugammadex use, the authors implemented a cognitive aid to help choose between reversal agents. The purpose of this study was to determine if sugammadex use changed after cognitive aid implementation. The authors’ hypothesis was that sugammadex use and associated costs would decrease.
A cognitive aid suggesting reversal agent doses based on train-of-four count was developed. It was included with each dispensed reversal agent set and in medication dispensing cabinet bins containing reversal agents. An interrupted time series analysis was performed using pharmaceutical invoices and anesthesia records. The primary outcome was the number of sugammadex administrations. Secondary outcomes included total pharmaceutical acquisition costs of neuromuscular blocking drugs and reversal agents, adverse respiratory events, emergence duration, and number of neuromuscular blocking drug administrations.
Before cognitive aid implementation, the number of sugammadex administrations was increasing at a monthly rate of 20 per 1,000 general anesthetics (P < 0.001). Afterward, the monthly rate was 4 per 1,000 general anesthetics (P = 0.361). One month after cognitive aid implementation, the number of sugammadex administrations decreased by 281 per 1,000 general anesthetics (95% CI, 228 to 333, P < 0.001). In the final study month, there were 509 fewer sugammadex administrations than predicted per 1,000 general anesthetics (95% CI, 366 to 653; P < 0.0001), and total pharmaceutical acquisition costs per 1,000 general anesthetics were $11,947 less than predicted (95% CI, $4,043 to $19,851; P = 0.003). There was no significant change in adverse respiratory events, emergence duration, or administrations of rocuronium, vecuronium, or atracurium. In the final month, there were 75 more suxamethonium administrations than predicted per 1,000 general anesthetics (95% CI, 32 to 119; P = 0.0008).
Cognitive aid implementation to choose between reversal agents was associated with a decrease in sugammadex use and acquisition costs.
Cognitive aids implemented in the perioperative environment are associated with an improvement in quality and patient safety.1–3 The most widely used cognitive aid is the surgical safety checklist, which has been shown to decrease morbidity and mortality associated with surgery in multiple settings globally.4 Another cognitive aid that has been instrumental in improving patient safety is the emergency manual, which has been shown to improve implementation of best known practices during critical events in various healthcare settings.2,5–7 While the mechanism behind the success of cognitive aids is not entirely understood, a recent study8 describes factors that are associated with the successful implementation of cognitive aids. Some of the factors include completing more implementation steps, leadership support, having an implementation champion, and institution-specific customization of the cognitive aid. Ultimately, one of the most important features of a cognitive aid is that it can result in improved practice processes.1,4
In April 2016, sugammadex was introduced in our institution. As its use and subsequent pharmacy acquisition costs increased, our department was asked to review neuromuscular blockade reversal practices with the intent to implement a practice change to decrease the use of sugammadex and associated costs. A review of a random sample of 87 anesthesia records demonstrated that 71% of patients received sugammadex more than 1 h after the administration of rocuronium (see Supplemental Digital Content 1, fig. S1, showing the sugammadex usage in August 2017; https://links.lww.com/ALN/C44). While duration from last dose of rocuronium does not determine the need for reversal of neuromuscular blockade, based on a previous study9 we estimated that more than half of the patients would likely have had sufficient recovery from neuromuscular blockade to allow for the administration of neostigmine with glycopyrrolate.
Given the previous use of checklists and emergency manuals to guide clinician behavior, and the request to implement a practice change to decrease the use of sugammadex, we sought to implement a cognitive aid to guide management of neuromuscular blockade reversal agents in the operating room setting. A previous study10 used a cognitive aid, “Neostigmine Dosing Guide,” to help change neuromuscular blockade reversal agent administration practices in an institution where only neostigmine was available at the time. We adapted the “Neostigmine Dosing Guide” cognitive aid to take into account sugammadex and the use of only peripheral nerve stimulators in our institution. The goal of our cognitive aid was to guide management of neuromuscular blockade reversal agents in an evidence-based, safe, and cost-effective manner.
The purpose of this study was to determine if implementation of the cognitive aid resulted in a change in sugammadex use in the operating room by performing an interrupted time series analysis. Our hypothesis was that the cognitive aid would decrease sugammadex use in the operating room, which would be associated with decreased total acquisition costs of neuromuscular blocking drugs and reversal agents while maintaining safe and timely patient care.
Materials and Methods
Setting
A waiver of informed consent for medical record review was obtained from the institutional review board. This study was conducted at an urban tertiary care, academic medical center with 415 inpatient beds. The institution performs more than 22,000 anesthetic cases annually, of which approximately 26% are general anesthetics utilizing neuromuscular blockade. Multiple clinical information systems are used throughout the institution, with Anesthesia Touch (Plexus Technology Group, LLC, USA) being the primary system used for anesthesia record documentation, and Pharmacy Clinical Workstation (Cerner Corporation, USA) being the primary clinical and drug distribution pharmacy platform that is integrated with point-of-care and financial systems. Two systems are in place for dispensing medications in the operating room: password-protected medication dispensing cabinets (Omnicell, USA), and manually retrieved standardized sets of medications directly from a pharmacist. For each dispensing system, both neuromuscular blockade reversal agents are simultaneously available for use by the anesthesia clinicians. This manuscript adheres to the guidelines of the Standard for Quality Improvement Reporting Excellence.
Study Intervention
A cognitive aid similar to a previously described model10 was developed to guide management of neuromuscular blockade reversal agents (fig. 1). Dosing strategies for sugammadex and neostigmine were made based on a review article11 that outlines available evidence on the topic of neuromuscular blockade and reversal. A “notes” section was included on the lower portion of the cognitive aid to remind clinicians of subtle details and certain exceptions. The cognitive aid was presented at a departmental conference on November 16, 2017, during which attending physicians, resident physicians, and nurse anesthetists offered feedback and suggestions for enhancements. The following day, a departmental email was sent to solicit additional feedback from clinicians who may have missed the conference. After making changes suggested by our clinicians, the cognitive aid was formally adopted by our department on November 27, 2017. In operating rooms with a medication dispensing cabinet, the cognitive aid was placed in the same bin as sugammadex and neostigmine, with the goal of prompting the clinician to use the cognitive aid when reaching for these drugs. In operating rooms with standard medication sets manually dispensed by the pharmacist, the cognitive aid was included with each medication set containing sugammadex and neostigmine.
Peripheral Nerve Stimulators
Before October 26, 2017, three different peripheral nerve stimulators were available: MiniStim* MS-IVB (Halyard Health, United Kingdom), EZstim II (Life-Tech, Inc., USA), and DigiStim 2 Plus (Neuro Technology, Inc., USA). There were several units of each device type, and there was no particular pattern in which they were distributed. Our perioperative administration purchased the EZstim* III (Halyard Health) peripheral nerve stimulators and placed them in all anesthetizing locations on October 26, 2017. The previously used peripheral nerve stimulators were then placed into storage to be used only in the event that the EZstim* III should be unavailable.
Data Sources
Data on the number of administrations of drugs and acquisition costs were collected from accounting data in the Pharmacy Clinical Workstation system. For every month, each vial or prefilled syringe that was removed and not returned to the pharmacy was counted as an administration, such that all drugs were counted regardless of dose used or wasted. The acquisition cost of each vial or prefilled syringe was obtained from the monthly pharmacy invoices. Total acquisition cost of neuromuscular blocking drugs and reversal agents was calculated by summing the individual cost of each drug counted every month, such that it reflects the total pharmacy invoice and takes into account different prices between outpatient and inpatient anesthetics. Costs differed between outpatient and inpatient anesthetics because the 340B Drug Pricing Program of the U.S. Department of Health and Human Services (Washington, DC) allows our institution to obtain discounted prices for outpatient drugs. The median per vial or prefilled syringe costs were as follows: sugammadex 200 mg vial, $89; neostigmine 3 mg syringe, $31; glycopyrrolate 1 mg vial, $12; rocuronium 100 mg vial, $4; vecuronium 10 mg vial, $5; atracurium 100 mg vial, $15; and suxamethonium 200 mg syringe, $35.
Anesthesia record data were documented in the electronic system Anesthesia Touch. Using the Practice Management Intelligence Reporting System (Plexus Technology Group, LLC, USA), data were extracted for all anesthetics documented during the study period. Specific data points extracted included names of adverse events, surgery end and emergence times (to calculate emergence duration), type of surgery, American Society of Anesthesiologists (ASA) physical status, surgery start and end times (to calculate surgery duration), patient age, airway management technique, and type of anesthesia. All cases documented as ASA physical status VI or VIE, cases that were cancelled on the day of surgery before induction, and those documented as only labor epidural analgesia were removed. Adverse events were filtered to only include adverse respiratory events (reintubation, respiratory insufficiency, difficult airway, bronchospasm, and laryngospasm), which are documented when they occur in the operating room or the postanesthesia care unit.
Primary and Secondary Outcomes
The primary outcome was the number of sugammadex administrations per 1,000 general anesthetics per month as assessed by interrupted time series analysis.
Secondary outcomes included the following:
Total acquisition costs of neuromuscular blocking drugs and reversal agents (rocuronium, vecuronium, atracurium, suxamethonium, sugammadex, neostigmine, and glycopyrrolate) per 1,000 general anesthetics per month
The number of adverse respiratory events per 1,000 general anesthetics per month
The emergence duration (median minutes from surgery end to emergence) in patients undergoing general anesthesia
The number of administrations of neuromuscular blocking drugs, including rocuronium, vecuronium, atracurium, and suxamethonium, per 1,000 general anesthetics per month
Sensitivity Analyses
We performed several sensitivity analyses to verify the robustness of our statistical models. We assessed three drugs not associated with neuromuscular blockade that should not have been affected by the cognitive aid but would be subject to the same threats to internal validity as the primary and secondary outcomes. We ran models adjusted for several risk factors for perioperative respiratory complications12–14 to evaluate if time-varying confounding affected the number of adverse respiratory events. We also assessed the number of supraglottic airway techniques and general anesthetics to evaluate for possible time-varying confounding on sugammadex use.
Statistical Analysis
We performed an interrupted time series analysis to investigate the effects of our cognitive aid on the number of drug administrations, total pharmaceutical acquisition costs, adverse respiratory events, and emergence duration. A formal statistical power analysis was not performed. Rather, we used previous research15 that suggested that data from at least 8 months before and after our intervention would be sufficient to estimate the level and slope parameters.
For outcomes that were monthly counts (e.g., number of drug administrations, number of adverse respiratory events, etc.), data were analyzed using segmented Poisson and Negative Binomial models. We fit, for each outcome, a previous specified model of = β0 + β1 · ts + β2 · Xt + β3 · tp, where is the predicted value at time t, β0 is the intercept, β1 is the baseline trend, β2 is the level change, and β3 is the trend change after the cognitive aid was implemented. The times ts and tp are the time in months (defined as 30-day periods, except for the first and last months, which were 15-day periods) from the beginning of the study and the time after cognitive aid implementation; Xt is a scalar (0 for precognitive aid, and 1 for postcognitive aid) as described in a previous study.16
Data for a 1-month period (November 16, 2017, to December 15, 2017) surrounding the initial implementation of the cognitive aid were excluded from the fitting of models. The count models included an offset term of the natural logarithm of the number of general anesthetics per month. Estimates of the parameters of interest (β0, β1, β2, β3) were presented from the model with the best fit based on the lowest corrected Akaike Information Criteria.
For emergence duration, which was not a count that was dependent on the number of anesthetics per month, we used a linear regression approach allowing autoregressive terms to enter the model if they improved model performance, which would account for the possible autocorrelation of the time series nature of these data. These linear regression time series selection models were also run on the monthly count data standardized by the number of general anesthetics per month as sensitivity analyses. These models were estimated using the SAS/ETS (version 14.1; SAS Institute Inc., USA) procedure “Autoreg” with maximum likelihood estimation. To account for autocorrelation, we used a stepwise selection to allow up to six lagged dependent variables to enter the model and calculated generalized Durbin–Watson statistics for the final model for each outcome.
Estimates of the postcognitive aid trend (β1 + β3) were also calculated by reparameterizing the above model. Resulting models were used to estimate mean predicted values and 95% CIs for each time point given two scenarios: first, assuming the cognitive aid was implemented, and second, under the counterfactual assumption that the cognitive aid had not been implemented, by setting Xt = 0 for the prediction. Estimates of the variances for the pre- and postpredictions were calculated from the span of the 95% CIs, and then used to estimate the variance of the difference, which was used to estimate the 95% CI for the estimated mean difference (precognitive aid minus postcognitive aid) at each time point. SAS version 9.04.01M3 was used with SAS Enterprise Guide (version 7.15HF3; SAS Institute Inc., USA) for the time series analysis.
The number of drug administrations and general anesthetics was summarized for the entire cohort for pre- and postcognitive aid periods. Standardized differences between pre- and postcognitive aid characteristics and corresponding 95% CIs were calculated using Hedge’s g for interval data and Cliff’s delta for ordinal and dichotomous data with the effsize function (effsize_07.1) running on R software (R version 3.5.3 [2019-03-11] through R Studio (R Studio Version 1.1.463; RStudio, Inc., USA).
Results
Between December 1, 2016, and July 31, 2018, there were a total of 12 time periods before and 8 time periods after the intervention that were included in the analysis. Table 1 reports the number of monthly general anesthetics and drug administrations.
Figure 2 presents the time series for sugammadex, total acquisition cost of neuromuscular blocking drugs and reversal agents, adverse respiratory events, and emergence duration, while figure 3 presents the times series for the neuromuscular blocking drugs, with the fitted pre- and postcognitive aid trends, as well as the extrapolated precognitive aid trends that were used to calculate the counterfactual differences.
The number of sugammadex administrations in the first month of the study period was 238 per 1,000 general anesthetics. Before cognitive aid implementation, the number of sugammadex administrations was increasing at a monthly rate of 20 per 1,000 general anesthetics (P < 0.001). After cognitive aid implementation, the monthly rate decreased by 16 sugammadex administrations per 1,000 general anesthetics (P = 0.032), with a postintervention monthly rate of four sugammadex administrations per 1,000 general anesthetics (P = 0.361). Implementation of the cognitive aid was associated with an immediate decrease of 281 sugammadex administrations per 1,000 general anesthetics in the postintervention month (95% CI, 228 to 333; P < 0.001). In the final study month, there were 509 fewer sugammadex administrations than predicted per 1,000 general anesthetics (95% CI, 366 to 653; P < 0.0001), and total pharmaceutical acquisition costs per 1,000 general anesthetics were $11,947 less than predicted (95% CI, $4,043 to $19,851; P = 0.003).
Implementation of our cognitive aid did not significantly affect the number of adverse respiratory events, emergence duration, or number of administrations of rocuronium, vecuronium, or atracurium. In the final month, there were 75 more suxamethonium administrations than predicted per 1,000 general anesthetics (95% CI, 32 to 119; P = 0.0008).
Sensitivity Analyses
Results of the sensitivity analyses are presented in Supplemental Digital Content 1, figure S2 (https://links.lww.com/ALN/C44) and Supplemental Digital Content 2, tables S1, S2, and S3 (https://links.lww.com/ALN/C45). Covariate adjustment did not improve model fit for any of the study variables. The estimated coefficients from the segmented regression analysis and the combined intercept and slope changes associated with the cognitive aid, expressed as counterfactual differences at 8 months postimplementation, are also found in Supplemental Digital Content 2, tables S1, S2, and S3 (https://links.lww.com/ALN/C45).
Discussion
In this interrupted time series analysis, we found that implementation of an evidence-based cognitive aid with criteria for selecting sugammadex versus neostigmine and providing dosing recommendations based on train-of-four count was associated with a decrease in the number of sugammadex administrations and total pharmaceutical acquisition cost of neuromuscular blocking drugs and reversal agents. We did not observe a clinically meaningful change in the number of adverse respiratory events or the emergence duration. We found a small, clinically insignificant increase in the number of administrations of suxamethonium, but not of rocuronium, vecuronium, or atracurium.
The means by which our cognitive aid exerted its effect to decrease the number of sugammadex administrations may be related to a variety of factors. First, the cognitive aid was immediately available to providers at the point of care. As part of the implementation process, our pharmacists placed the cognitive aid in the same bin as sugammadex and neostigmine in medication dispensing cabinets and dispensed the cognitive aid in conjunction with each medication set containing sugammadex and neostigmine. While this approach facilitated easy access to the cognitive aids, this strategy alone would likely have been insufficient.
Several studies have demonstrated that implementation of cognitive aids is a complex process, which results in varying levels of success depending on how the implementation process occurs.8,17,18 An implementation champion has been identified17 as an important component of successful implementation of cognitive aids. In our department, the role of the implementation champion was to develop the cognitive aid, solicit feedback from members of the department, and oversee the printing and appropriate distribution of the cognitive aid. In addition, the role included reminding clinicians about the cognitive aid at departmental meetings, during which the rationale behind implementation of the cognitive aid was explained and demonstrations of how to use the cognitive aid were performed, both of which function to build understanding and buy-in from department members.17 Furthermore, there was leadership support from the department chair, which primarily took the form of encouraging the implementation champion and providing dedicated time to train staff in the use of the cognitive aid, both of which further facilitate use of the cognitive aid.8,18 Importantly, while the implementation champion and the department chair promoted adoption of the cognitive aid, there was no financial incentive or other pressure (e.g., evaluation of individual use as a clinical performance indicator)10 to influence use of the cognitive aid.
Second, we designed and customized the cognitive aid to be user-friendly within our institutional context: Neuromuscular blockade reversal agent choice and dosing recommendations were based on qualitative train-of-four count and fade. While a more comprehensive assessment of neuromuscular blockade would include assessment of a train-of-four ratio, we intentionally did not include these criteria because our institution does not have a quantitative train-of-four monitor. Furthermore, we hoped that the cognitive aid would add value to the anesthesia clinician in the management of neuromuscular blockade reversal. At the time sugammadex was initially added to the hospital formulary, sugammadex was a relatively new drug unfamiliar to our clinicians with some uncertainty regarding its clinical use. Several studies have emphasized that simplicity of design encourages and promotes use,19,20 and that customization makes cognitive aids more relevant to the local institution.3
Finally, we used a multidisciplinary and collaborative approach to create the cognitive aid: A departmental meeting was held in which the idea for the cognitive aid was introduced, an email soliciting comments was sent to all department members, and feedback was integrated before implementation of the final version. In addition, we invited one of the pharmacists to give feedback on the cognitive aid and discuss strategies for implementation. Several studies have identified that such an approach improves cognitive aid implementation because it gives clinicians an opportunity to be engaged and become part of the adoption and implementation process.1,8
While a common quality improvement approach to achieve drug prescribing practice changes among clinicians is the audit-feedback cycle, we did not choose this method because it is time- and resource-intensive, its effects are generally small, and changes are often not sustained.21 For example, Todd et al.22 found that audit-feedback cycles combined with the provision of peripheral nerve stimulators in all operating rooms increased the rate of appropriate use of neuromuscular blockade reversal agents and decreased the rate of residual neuromuscular blockade and reintubations. However, after cessation of audit-feedback cycles, the authors found an increase in inappropriate use of neuromuscular blockade reversal agents.23 Similarly, Colombet et al.24 found that while audit-feedback cycles decreased the rate of intravenous proton pump inhibitors, usage patterns eventually returned to baseline when audit-feedback cycles were no longer in place.
Concurrent with decreased sugammadex use, we found a decrease in the total acquisition costs of neuromuscular blocking drugs and reversal agents. To determine if the changes in neuromuscular blockade reversal agent usage patterns resulted in any negative impact, we studied the number of adverse respiratory events and the emergence duration, as an increase in these outcomes would likely negate any cost savings associated with decreased sugammadex use. Significant changes for adverse respiratory events were not observed in any of the parameters of the segmented regression analysis or the counterfactual differences at 8 months postimplementation. Similarly, we did not observe an increase in the emergence duration. While the overall number of adverse respiratory events was small and additional factors could be considered, implementation of the cognitive aid did not appear to have a readily observable negative effect on the number of adverse respiratory events or emergence duration.
While our cognitive aid contains some basic information about reversal of neuromuscular blockade to guide clinicians, there are several elements that could be added to enhance the cognitive aid depending on institutional practices and resources. A note indicating that the ideal location for measuring the train-of-four count is the adductor pollicis25,26 could be added in the “notes” section. Another useful reminder may be to indicate that greater doses of sugammadex are required for reversal of vecuronium-induced compared to rocuronium-induced neuromuscular blockade given the greater neuromuscular blocking potency of vecuronium and lesser affinity of sugammadex for vecuronium.27 Finally, while we did not include train-of-four ratio monitoring in our cognitive aid because we do not have quantitative monitoring available in our institution, the addition of this information should be considered if quantitative monitoring is available, because even protocolized administration of neostigmine does not guarantee complete reversal of neuromuscular blockade.28,29 For an example of such a cognitive aid, see Supplemental Digital Content 3 (https://links.lww.com/ALN/C46), a modified version of the Neuromuscular Blockade Reversal Cognitive Aid.
Our study has several important limitations. First, the interrupted time series statistical design does not allow for conclusions of causality, and it is unclear if confounding factors could have influenced the outcomes. For example, knowing that the leadership is observing the pharmaceutical usage patterns (the Hawthorne effect30 ) could have itself contributed to a change in practice patterns. Second, each operating room was supplied with a single type of qualitative peripheral nerve stimulator approximately 1 month before the cognitive aid was implemented. While the increased availability of a single type of peripheral nerve stimulator could have affected the usage patterns of neuromuscular blockade reversal agents, previous studies22,31 have found that availability of peripheral nerve stimulators alone is not sufficient to affect practice changes. In addition, given the retrospective nature of our data, it was not possible to accurately determine how often the peripheral nerve stimulators were used. Finally, while our cognitive aid cards could have sustainable effects, over time they could get damaged or lost. Long-term effects of the cognitive aid are unclear. An electronic-based cognitive aid, such as the electronic preanesthetic induction patient safety checklist used by Wetmore et al.,19 might be more sustainable as it would not be lost or destroyed over time unless intentional changes in software are made.
In summary, through this interrupted time series analysis, we observed that the implementation of an evidence-based cognitive aid was associated with a decrease in the number of sugammadex administrations and total acquisition costs of neuromuscular blocking drugs and reversal agents, without an increase in adverse respiratory events or emergence duration. These findings could be useful to institutions that are considering adding sugammadex to their formulary and for those that already have sugammadex. Some institutions may choose not to introduce sugammadex because of previous studies showing that its adoption results in tripling of pharmaceutical acquisition costs associated with neuromuscular blockade.32 However, the adoption and implementation of a cognitive aid could help clinicians using an evidence-based approach determine when to use sugammadex versus neostigmine in a cost-effective manner. If a cognitive aid is to be used at institutions with quantitative monitoring devices available, the modified version should be considered because quantitative train-of-ratio monitoring would offer a more thorough protocol for reversal of neuromuscular blockade. Nevertheless, it is important to emphasize that the implementation process of the cognitive aid is critical to its successful use.
Acknowledgments
We would like to thank Saul Weingart, M.D., Ph.D., Department of Medicine, Tufts Medical Center, Boston, Massachusetts, for reviewing the manuscript and offering suggestions for its improvement. We would also like to thank Robin Ruthazer, M.P.H., Institute for Clinical Research and Health Policy Studies, Biostatistics, Epidemiology, and Research Design Center, Tufts Medical Center, Boston, Massachusetts, for performing the statistical analysis.
Research Support
Support for this study was provided by a Tufts Medical Center Insurance Company Risk Management Grant, and the Tufts Clinical and Translational Science Institute, Boston, Massachusetts, which is funded in part by the National Center for Advancing Translational Sciences, National Institutes of Health, Bethesda, Maryland, award No. UL1TR002544.
Competing Interests
The authors declare no competing interests.