Opioid Stewardship Program and Postoperative Adverse Events: A Difference-in-differences Cohort Study

The roots of the United States opioid crisis are complex and frequently attributed to overprescribing of opioid pain medications, aggressive pharmaceutical marketing, underappreciation of the dangers of opioids and risks of misuse, and overemphasis of the “fifth vital sign.”¹  Although public awareness of opioid-related risks is increasing and opioid prescribing is in decline, overdose death rates from prescription and illicit opioids remain unacceptable. In parallel, increased rates of mental illness, social and economic marginalization, access to synthetic drugs, and an increased incidence of chronic pain across the population highlight the critical need for up-to-date and innovative educational efforts surrounding safe opioid prescribing and multimodal pain management strategies for all healthcare disciplines.^2,3 

In response to the opioid and pain management crisis, the Centers for Medicare and Medicaid Services (Woodlawn, Maryland) agreed to support a Premier, Inc. (Charlotte, North Carolina) initiative to implement strategies to decrease opioid use in hospitals. Premier then partnered with the American Society of Anesthesiologists (Schaumburg, Illinois) to conduct a pilot program through the Centers for Medicare and Medicaid Services Hospital Improvement Innovation Network entitled, “Safer Postoperative Pain Management: Reducing Opioid-related Harm,” to test a multidisciplinary approach to measurably reduce and/or prevent opioid-related harm among selected types of adult hospitalized patients. Invited speakers from the American Society of Anesthesiologists who specialize in pain medicine in conjunction with Premier’s Hospital Improvement Innovation Network education workstream developed a 6-month pain education/opioid stewardship curriculum.

The purpose of the pilot program was to evaluate the impact of this education intervention on opioid-related adverse drug events in an intervention hospital group (n = 32) versus matched nonintervention control hospitals (n = 33). Secondary outcomes evaluated were postoperative opioid use and assessment of measures of opioid best practice guidelines of patients receiving opioid therapy. We hypothesized that hospitals that were part of the intervention would demonstrate lower opioid-related adverse events (primary outcome), reduced opioid use after selective surgical procedures, and improved documentation of opioid related practice (secondary outcomes) compared to nonintervention hospitals.

The study was approved by the Institutional Review Board of Rush University Medical Center (Chicago, Illinois; 19061003-IRB1). This article was prepared using the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines.⁴  The study was granted waiver of informed consent because it evaluated existing deidentified records, was not greater than minimal risk, and was deemed to be Health Insurance Portability and Accountability Act–compliant because safeguards were in place to protect the personal health information of the subjects.

The study was a difference-in-differences cohort design of adverse opioid events and opioid utilization before and after a performance improvement education intervention in network hospitals that participated in the intervention versus matched hospitals that did not receive the intervention. Preinterventional data were obtained retrospectively, and postinterventional data were collected prospectively. The primary hypothesis was that the incidence of opioid adverse events would be reduced in the hospitals that participated in the educational intervention compared to the nonintervention hospitals. The secondary hypothesis was that measures of postoperative use after colorectal, hip, and knee arthroplasty would be reduced and that documentation of opioid best practice guidelines would be observed in the intervention compared to the nonintervention hospitals.

Thirty-two network hospitals were selected to participate in this 6-month pilot opioid stewardship education program from October 2017 to March 2018. A nonintervention cohort of 33 network hospitals was selected by informatics personnel from the pool of network sites not involved in the pilot intervention of similar bed count (±100), location (census divisions), critical access status (yes/no), academic affiliation (yes/no), and visits in the same time period as the study sites. The total number of colorectal, hip, and knee arthroplasties had to be within 5% of the total surgical volume of the intervention site of the same demographics. The nonintervention hospitals followed standard of care and were unaware of their participation in the study.

The intervention was developed in collaboration with anesthesiologists who specialize in the management of acute and chronic pain. The core intervention was a care bundle comprising six evidence-based webinar presentations of best practice guidelines for opioid analgesia use. The topics of these presentations were (1) safer opioid medicine guidelines to decrease overuse, misuse, and inappropriate prescribing; (2) preoperative surgical screening and opioid medication management for surgical procedures including nerve blocks; (3) multimodal pain management strategies and documentation; (4) postoperative pain and sedation screening, safe use of patient controlled analgesia, technology, the role of nursing response, and handoffs in patients receiving opioids; (5) management of postoperative pain including discharge planning, storage, and disposal of opioids; and (6) overview of addiction, basis of screening, interventions and referrals for treatment, and patient and family engagement for prevention of opioid misuse. These educational presentations were supplemented by three additional activities: (1) on-site education and technical assistance, office hours, and coaching calls; (2) tools and resources, including monthly performance reports, evidence-based guidelines, etc.; and (3) patient and family engagement sessions in care planning.

Employing a rapid Plan-Do-Study-Act cycle design, the intervention used a multidisciplinary approach to measure reduction and/or prevention of opioid-related harm among adult hospitalized patients, opioid use in patients undergoing elective hip or knee arthroplasty or colectomy procedures, and adherence to best practice guidelines for opioid administration. Participating hospitals were required to identify multidisciplinary teams to lead the work including one anesthesiology champion. Teams were composed of a range of clinical and nonclinical staff, including hospital executive leadership, quality and nursing leadership and staff, emergency medicine and anesthesiology clinicians, pharmacists, and staff working in regulatory compliance and patient safety.

Opioid-related adverse events, measures of postoperative opioid utilization, and opioid use best practice procedures were selected before the intervention. Three sentinels of opioid related adverse events were selected: (1) opioid overdose, wrong substance given or taken in error, (2) naloxone administration, and (3) respiratory failure causing prolonged ventilation. Opioid overdose, wrong substance given or taken in error rates were determined by number of hospitalized patients with a secondary International Classification of Diseases, Ninth Revision or Tenth Revision diagnostic code (International Classification of Diseases, Ninth Revision: 965.09, E850.2 or International Classification of Diseases, Tenth Revision: T40.2X*A, TX40.4X*A, T40.60*A, TX40.68*A, TX40.0X*A) that was not present on admission of the number of acute care inpatients 18 yr or older discharged during the same period. The accuracy of using International Classification of Diseases, Ninth Revision or Tenth Revision codes to assess opioid overdose, wrong substance given or taken in error using these diagnostic codes has been demonstrated to be 96.3%, with a positive predictive value between 76 and 81%.^5,6  Naloxone reversal rates were determined by the number of inpatients administered naloxone out of the number of acute inpatients who received an opioid analgesic during their hospital stay. Patients treated with naloxone in the emergency department or for the treatment of nausea (International Classification of Diseases, Ninth Revision: 787.01, 787.02 or International Classification of Diseases, Tenth Revision: R11.0, R11.2) or pruritus (International Classification of Diseases, Ninth Revision: 698* or International Classification of Diseases, Tenth Revision: L29*) were excluded. Respiratory failure was defined as postoperative respiratory failure as a secondary diagnosis, prolonged mechanical ventilation or reintubation in elective surgical patients 18 yr of age or older. Excluded are respiratory failure as a primary diagnosis; acute respiratory failure on admission; tracheostomy as the only operating room procedure or tracheostomy before the first operating room procedure; neuromuscular disorders; laryngeal, oropharyngeal, or craniofacial surgery involving significant risk of airway compromise; craniofacial anomalies that had a procedure for the face, esophageal resection, lung cancer, lung transplant, or degenerative neurologic disorders; cases with respiratory or circulatory diseases; and obstetric discharges. The incidence was determined by the number of postoperative patients with a diagnostic code of respiratory failure (International Classification of Diseases, Tenth Revision-Procedure Coding System: ACURF2D*, PR9672P*, PR9671P*, and PR9604P*) of the total number of elective surgical discharged patients after an operating room procedure. These specifications are in line with those defined per the Agency for Healthcare Research and Quality (North Bethesda, Maryland) Quality Indicators Version 6.0.⁷  Opioid adverse events were extracted from the electronic medical records starting September 2016 and ending October 2018.

Postoperative opioid utilization assessment was evaluated in adult (more than 18 yr of age) patients undergoing elective colorectal surgery and hip and knee arthroplasty. These procedures were selected because the majority of these patients receive an opioid analgesic throughout their hospital stay, the daily morphine milligram equivalent is clinically important, and each of these surgeries has been shown to benefit from multimodal analgesia regimens. Utilization parameters selected for evaluation included (1) the number of days of opioid therapy per 100 patient days was determined among adult inpatients with a primary International Classification of Diseases, Ninth Revision or Tenth Revision Procedure Coding System code related to the specific procedure; (2) the average daily opioid administered in morphine milligram equivalent was determined from the total morphine milligram equivalents administered divided by the total number of adult patients with a primary International Classification of Diseases, Ninth Revision or Tenth Revision Procedure Coding System code related to the specific procedure and had an opioid administered; (3) the average daily oral opioid dose in morphine milligram equivalents was determined from the total morphine milligram equivalents divided by the total number of adult patients with a primary International Classification of Diseases, Ninth Revision or Tenth Revision Procedure Coding System code related to the specific procedure and an oral opioid administered; (4) the incidence of patients administered greater than 90 morphine milligram equivalents of the adult inpatients with a primary International Classification of Diseases, Ninth Revision or Tenth Revision Procedure Coding System code and opioid administered; and (5) the incidence of patients administered orally greater than 90 morphine milligram equivalents of the adult inpatients with a primary International Classification of Diseases, Ninth Revision or Tenth Revision Procedure Coding System code and an oral opioid administered. Opioid utilization assessments were extracted from medical records starting September 2016 and ending October 2018.

Process assessments of opioid best practice guidelines were performed by study staff at each participating facility. Medical records were evaluated for documentation of preoperative assessment of obstructive sleep apnea, assessment for opioid tolerance, patient involvement in the pain management plan, evidence of multimodal analgesia use, pain assessment using a standardized tool at least every shift in patients receiving opioids, assessment of sedation using a standardized tool at least every shift in patients using opioids, and written and verbal discharge instructions that included the opioid education bundle. Chart audits were performed the month before the start of the intervention (September 2017) and for 6 consecutive months, October 2017 to March 2018, after the start of the program. Ten percent of the patients in the targeted surgery groups (colorectal, hip, and knee arthroplasty) to a maximum of 10 per site were randomly selected by the audit team and evaluated.

Data for opioid related adverse events and opioid utilization for both the interventional sites and noninterventional cohort were extracted from the electronic medical records by informatics personnel, deidentified, and aggregated into monthly summary statistics for each month of the study per hospital site. No individual patient information was included in the bundled datasets. Hospitals were coded so that the site location could not be identified by the study investigator performing the data analysis (R.J.M.). Results of chart audits of best practice guidelines were also provided as deidentified monthly aggregated values per study site. Informatics personnel examined the dataset for outliers and missing data before release. Data extraction for the analysis used in this report was performed on April 9, 2019.

The primary outcome of interest was the monthly incidence of opioid-related adverse events. Counts of opioid adverse events were divided by the total number of records reviewed per month and were normalized to 10,000 patients for International Classification of Diseases, Ninth Revision or Tenth Revision codes indicating an opioid-related overdose, wrong substance given or taken in error, respiratory failure rates, and naloxone use. Secondary outcomes included opioid utilization measures after total hip and knee arthroplasty and colorectal surgery as well as adherence to best practice guidelines for opioid utilization. The distribution of the rates of opioid adverse events and opioid utilization measures did not meet the criteria for departure from normality (Kolmogorov–Smirnov test), and q-q plots, and the differences-in-differences between control and intervention participants for opioid adverse events and opioid utilization data were evaluated using a generalized linear multiple regression model with an identity link function. The fitted model was of the form Ŷ_it= β₀+ β₁*group + β₂*int + β₃*(group*int) + β₄*t_s+ ε_it where β₀ is the intercept, β₁ and β₂ are the coefficients for group and intervention (int), and β₃ is the difference-in-differences between groups before and after the intervention. The common slope is β₄ and t_s is the time in months from the start of the data collection period. Time was included in the difference-in-differences model to control for aggregated time effects due to the extended collection period.⁸  Scalar variables were used for group (0 control and 1 participating sites) and int (0 preintervention and 1 postintervention). The effect size of the difference-in-differences is reported as the coefficient β₃ and the 99% CI of the difference.

Adherence to best practice guidelines based on chart audits among Premier sites participating in the intervention was analyzed using a linear mixed model with a first-order autoregressive covariance structure for the repeated effects (time). The facility code and the time from the control (September 2018) were considered random effects. Differences in estimated marginal means and 99% CI are reported for each month after the intervention compared to the control month. Standardized differences (99% CI) between opioid intervention and nonopioid intervention facilities were calculated as Hedge’s g for interval data and Cliff’s delta for ordinal and dichotomous data. This report is the primary analysis of the data from this study. The data analysis plan as well as the primary and secondary outcomes were made a priori before accessing the data. The three measures of opioid adverse events and the five measures of opioid use were not independent measures; therefore, a P < 0.01 was selected to reject the null hypothesis for the three comparisons of the primary outcome and the five comparisons of the secondary to obtain a family wide error rate of 0.029 and 0.049, respectively. Post hoc comparisons of adherence to best practice guideline were adjusted for six comparisons, P < 0.008. All analyses were performed two-tailed. Data analysis was performed using RStudio version 1.2.1335 (RStudio: Integrated Development for R. RStudio, Inc., Boston, Massachusetts; http://www.rstudio.com/) and R version 3.6.0, release date April 26, 2019 (The R Foundation for Statistical Computing, Vienna, Austria).

The study sample was based on all patients available at the intervention sites and a similar number of nonintervention sites for opioid overdose, wrong substance given or taken in error. Difference-in-differences of 25% in the preintervention incidence of opioid adverse events and opioid utilization were considered to be of clinical significance. Assuming 64 sites (32 per group) with each site examining 900 discharges per month, over the 26-month period of evaluation approximately 1.67 million records would be examined. A sample of 315,224 (157,612 per group) would have 90% power to detect a difference-in-differences in opioid overdose, wrong substance given or taken in error per 10,000 admissions of –0.1 in a 2 repeated-measures design having an autoregressive correlation type 1 covariance structure, when the SD of a single observation equal to 10 and the correlation between observations at the same site (Rho) equals 0.5 and an alpha of 0.01. Sample size calculations were made using PASS version 15.0.9, power analysis and sample size software (2017; NCSS, LLC, USA).

Thirty-two of the 490 (6.5%) Premier Hospital Improvement Innovation Network hospitals across 11 states (Florida, Iowa, Maryland, Massachusetts, New Jersey, New York, North Carolina, Ohio, Pennsylvania, Texas, and Virginia) were enrolled in the pilot opioid stewardship initiative; one hospital withdrew due to competing organizational priorities, with 31 hospitals completing the study. Thirty-three hospitals that did not receive the intervention and followed standard of care served as control facilities. Aggregate data for all sites receiving the intervention and the nonintervention sites were available for all study months. There were no missing data. Standardized differences between the intervention and nonintervention hospitals were negligible (less than 0.1) except for bed count, which was small (0.2 to 0.5; table 1).

The difference-in-differences analysis for opioid adverse events is shown in table 2. A total of 1,380,000 discharges were evaluated for opioid overdose, wrong substance given or taken in error, 590,000 in the intervention and 790,000 in the nonintervention group. The overall incidence of a new opioid overdose, wrong substance given or taken in error overdose, wrong substance given or taken in error was 1.0 per 10,000 discharged patients, and the difference-in-differences rate was –0.23 (99% CI, –1.10 to 0.64, P = 0.499) between the intervention group and control. A total of 157,000 surgical discharges were evaluated for respiratory failure, 62,000 in the intervention and 95,000 in the nonintervention group. The difference-in-differences incidence of respiratory failure per 10,000 elective surgical cases was lower (–13.6, 99% CI, –29.0 to 0.0, P = 0.028) in the intervention compared to the nonintervention group.

A total of 867,000 patients receiving opioids were evaluated for naloxone use, 360,000 in the intervention and 507,000 in the nonintervention group. Preintervention naloxone use per 10,000 patients receiving an opioid was statistically significantly less in the intervention group, difference –17 (95% CI of the difference, –28 to –6, P < 0.001), but there was no significant difference in the postintervention period, difference 2 (99% CI of the difference, –15 to 11, P = 0.702); however, there was an increase in the difference-in-differences incidence of 15.2 (99% CI of the difference, 3.8 to 30.0, P = 0.011) postintervention in the intervention group, suggesting an effect of nonparallelism in the difference-in-differences estimate. The effect of nonparallelism in the difference-in-differences estimate is further supported by the lack of difference in the postintervention slopes between the intervention and nonintervention groups (P = 0.368).⁹ 

Difference-in-differences and the common slope for the time effect of opioid utilization assessments are shown in table 3. In colorectal surgery, there were no statistically or clinically significant differences-in-differences in the days of opioid therapy, the average daily opioid use in morphine milligram equivalent, or the fraction of patients receiving daily high dose (greater than 90 morphine milligram equivalents) opioid analgesics. In hip and knee arthroplasty, the difference-in-differences postintervention in days of opioid therapy per 100 patient days was greater in the intervention group compared to the controls, but this difference is not likely clinically important. The differences-in-differences in the other measure of opioid use were not statistically different after hip or knee arthroplasty.

After knee arthroplasty, the common time slope of the average daily total morphine milligram equivalent and the average daily oral morphine milligram equivalent for the combined intervention group and control group was less than 0 (fig. 1). In the combined intervention and control group, the average total daily opioid morphine milligram equivalent decreased by 5.4 (99% CI, 4.3 to 6.2, P < 0.001), and the average oral daily morphine milligram equivalent decreased by 4.2 (95% CI, 3.9 to 4.5, P < 0.001) over the last 3 months of the study compared to the first 3 months (fig. 1); however, these changes are not likely clinically important. Figures for days of opioid use out of 100 patient days (Supplemental Digital Content, fig. 1, https://links.lww.com/ALN/C288), total daily opioid use (Supplemental Digital Content, fig. 2, https://links.lww.com/ALN/C345), fraction of total daily opioid use greater than 90 morphine milligram equivalents (Supplemental Digital Content, fig. 3, https://links.lww.com/ALN/C289), total daily oral opioid use (Supplemental Digital Content, fig. 4, https://links.lww.com/ALN/C290), and fraction of total oral daily opioid use greater than 90 morphine milligram equivalents (Supplemental Digital Content, fig. 5, https://links.lww.com/ALN/C291) are available in the online supplement.

Chart audits of opioid best practice guidelines are shown in table 4. Evidence of multimodal analgesia use was evident in 1,499 of 1,638 (91%, 99% CI, 90 to 94%) of the audited records. Best practice guidelines were increased from the baseline month in 37 of 42 (88%, 99% CI, 69.7 to 95.9%) of the observations and by greater than 10% for 24 of 47 (57%, 99% CI, 36 to 76%) of the observations. Nevertheless, only the documentation of pain assessments using a standardized tool at least every shift was increased by statistically significant amount compared to the month before the intervention for only 4 of the 6 months after the intervention.

This study demonstrates the lack of statistically important differences in opioid-related adverse events in the 31 U.S. hospitals that completed the opioid stewardship educational intervention compared with nonintervention hospitals. In addition, opioid administration outcomes and documentation of best practice measures of opioid use were not clinically significantly improved after the interventional group. Most importantly, our findings suggest that despite opioid and multimodal analgesia awareness, limited-duration educational interventions do not substantially change the clinical use of opioid analgesics in hospitalized patients in the postoperative period.

Although we were unable to demonstrate statistically robust differences-in-differences with high certainty in the incidence of opioid adverse events, our findings do merit some important clinical considerations. The rate of a secondary diagnosis of an opioid overdose, wrong substance given or taken in error event in patients observed in our study, 1 per 100,000 discharges, is substantially below the rates of 12 to 20 per 100,000 admissions with a primary diagnosis of opioid overdose, wrong substance given or taken in error, and suggest that opioid overdose, wrong substance given or taken in error in hospitals is not likely contributing substantially to opioid-related morbidity and mortality in the United States.^10,11  A health system–wide stewardship program to decrease opioid adverse events also failed to statistically decrease the number of opioid overdose events but found a downward slope after the intervention similar to the difference-in-differences estimate in the current study.¹²  The incidence of postoperative respiratory failure in this study, between 35 to 46 per 10,000 surgical discharges, was also substantially lower than the 140 per 10,000 surgical discharges found in the Department of Veterans Affairs (Washington, D.C.) National Surgical Quality Improvement Program data.¹³  The increase in the use of naloxone found in our differences-in-differences analysis in the intervention group was an unexpected finding of our study, but this difference may in part be explained by the lack of parallelism between the intervention and nonintervention groups in the period before the intervention. Nevertheless, we cannot rule out that heightened awareness of opioid-related respiratory depression after the intervention led to increased clinical use of naloxone.

After colorectal surgery, there was little reduction in the difference-in-differences for total or oral opioid use; however, the fraction of patients receiving high daily opioid doses (greater than 90 morphine milligram equivalents) was reduced by a clinically important amount after the intervention. The statistical increase in the number of patient days out of 100 patient days for opioid therapy for the orthopedic procedure appears to be driven by a sudden and sustained drop in the number of days per 100 patient days in the nonintervention hospitals beginning in July 2017, rather than an increase in the intervention group (Supplemental Digital Content, fig. 1, https://links.lww.com/ALN/C288). Nevertheless, these small differences are not likely associated with the intervention. The differences-in-differences in average daily and high daily amounts of opioids in orthopedic surgery were extremely modest, well below the clinically important threshold used in this study. The lack of sustained improvement in physician documentation of opioid best practice guidelines is not that surprising. Previous studies have shown that improved documentation after an educational intervention often requires implementation of a continuous quality improvement initiative.^14,15 

Previous studies of educational interventions targeting opioid use have demonstrated an effect primarily in discharge and outpatient opioid prescribing both in numbers of opioid prescriptions and amounts of opioid prescribed.^12,16  Unlike these studies, we focused on hospital use of opioids after an educational intervention and found a negligible to small impact of the program. In addition, our intervention was of a limited duration. Previous studies have shown that to have a significant impact on medical care process and patient outcomes, educational programs should be interactive and include auditing and feedback, reminders, and other types of clinician outreach, as programs that include only clinical practice guidelines and didactic lectures from opinion leaders have been shown to be less effective than when these methods are combined.¹⁷  An additional reason for the lack of effectiveness of the program may have been the implementation of standardized orders sets or potentially enhanced recovery from surgery programs at the hospital involved in this study that could make it more difficult to measure small changes in opioid use.

The National Pain Strategy, released in 2016 in response to the 2011 Institute of Medicine (Washington, D.C.) report “Relieving Pain in America,” developed a comprehensive strategy to address critical gaps in pain prevention, treatment, management, education, and other areas.³  Despite considerable momentum to implement the National Pain Strategy recommendations and the establishment of various local, regional, and national expert panels and working groups, much work and research are needed to guide how pain management principles and best practices are taught, implemented, and assessed. Preclinical and postgraduate education in the United States in multimodal pain management as well as opioid use disorder prevention, screening, and treatment are lacking.^18,19  Pain education reform (unfortunately with a focus primarily on substance use disorder prevention and treatment, and less of a focus on multimodal pain management) in U.S. medical schools is slowly evolving,^20,21  with national efforts to improve access to pain education materials for healthcare students in development such as by the 11 designated National Institutes of Health (Bethesda, Maryland) Pain Consortium Centers of Excellence in Pain Education.²² 

Our findings support the need for more longitudinal studies to evaluate the long-term impact of opioid stewardship education programs aimed at opioid utilization best practices and multimodal pain management strategies.^12,23,24  To evaluate the impact of these programs, outcomes should not only include opioid-related adverse events, hospital utilization, and discharge opioid prescribing rates, but also focus on additional outcome measures to ensure effective and safe pain management strategies. These measures should include patient-centered measures such as physical function and pain interference; patient compliance and adherence to multimodal analgesia; healthcare providers’ attitudes and knowledge about pain management; rates of chronic opioid prescribing in opioid-naive patients after surgery; rates of the development of chronic pain; screening for active or new opioid use disorders or medication misuse, etc. Measures of patient satisfaction, on the other hand, have been faulted for fueling unnecessary opioid prescribing.²⁵ 

The results of our study should only be interpreted in the context of its limitations. There are a few differences and potentially some overlap among the three-sentinel assessment used in this study. Opioid overdose, wrong substance given or taken in error was determined among all hospital discharges, and there was no requirement for mechanical ventilation, intubation, or reintubation. The assessment of opioid adverse events such as opioid overdose, wrong substance given or taken in error may not be a sensitive indicator of overuse or misuse of opioids especially in postoperative patients, despite the use of this metric in previous studies aimed at improving opioid utilization.¹²  Respiratory failure was only considered after a surgical procedure and was not specifically tied to overdose, wrong substance given or taken in error. Naloxone administration was determined among any patient receiving an opioid, and naloxone administration in patients with pruritus and nausea was excluded. Although we excluded patients who received naloxone for pruritus or nausea, we are unable to document the specific reasons for naloxone administration. This may have resulted in undercounting of patients who had pruritus or nausea treated with naloxone who later received naloxone for respiratory depression. The primary and secondary outcomes assessed in this study were extracted from hospital medical records, and despite the relatively high positive predictive value of these coded variables with correctly assigned outcomes, our estimates may be biased due to coding errors. Our data were obtained in monthly aggregates, and we were unable to assess patient-specific risk factors such as preoperative opioid use, widespread pain, chronic pain, age, or sex on postoperative opioid use. We did not consider the implementation of prewritten order sets or enhanced recovery from surgery pathways that may have been in use at the hospitals in this study, which may have affected our ability to measure differences in opioid orders for administration. We did not assess opioids prescribed or the amount prescribed at discharge. Our postintervention assessment period was only 6 months beyond the end of the pilot program; thus, long-term sustainability of the intervention was not measured. We did not include patient-centered outcomes such as physical function and pain interference as well as duration of postsurgical opioid use, and documentation of patient education measures on safe opioid use and tapering instructions. We applied a single educational intervention across a wide range of hospitals of various sizes and locations and did not attempt to customize the program to the specific needs and practice patterns of the individual institutions. Finally, the intervention and nonintervention hospitals in this study were assigned and not allocated randomly.

This opioid stewardship educational intervention targeting pain management did not have an impact on clinical outcomes in patients receiving opioid therapy in the hospital setting. Our findings suggest that despite opioid and multimodal analgesia awareness, limited-duration educational interventions do not substantially change the postoperative clinical use of opioid analgesics. Nevertheless, anesthesiologists should be the champions of local and national opioid stewardship efforts, and additional studies on the impact of these programs on postoperative opioid use as well as clinicians’ attitudes and prescribing practices are warranted to evaluate the benefit of these educational strategies.

The authors acknowledge Ashley Walton, J.D., American Society of Anesthesiologists, Washington, D.C., who provided strategic coordination and logistical support for the pilot study; Alex Hannenberg, M.D., F.A.S.A., American Society of Anesthesiologists, Schaumburg, Illinois, who provided subject matter expertise for pilot design and implementation and support for hospital participant inquiries; Tom Miller, Ph.D., M.B.A., Center for Anesthesia Workforce Studies, American Society of Anesthesiologists, Schaumburg, Illinois, who performed data analysis; Marty Buie, M.A., Premier, Inc., Charlotte, North Carolina, who provided data management oversight for the project; Michael Harry, Premier, Inc., who provided data management support and performed data preparation for analysis; and Emily Bartlett, R.H.I.A., M.B.A./H.C.M., Premier, Inc., who provided strategic coordination and logistical support for the pilot study.

The analyses upon which this publication is based were performed under Contract No. HHSM-500-2016-00075C, entitled “Hospital Improvement Innovation Network,” sponsored by the Centers for Medicare and Medicaid Services, Department of Health and Human Services. The American Society of Anesthesiology and the members who participated in this study received no compensation for participation.

Dr. Elkassabany has a financial relationship with Foundry Therapeutics (Menlo Park, California), a start up company that is concerned with local anesthetic formulation. The other authors declare no competing interests.