An Automated Software Application Reduces Controlled Substance Discrepancies in Perioperative Areas

Controlled substance diversion and tracking have received increased regulatory focus throughout the United States, in part because of the opioid epidemic. Within our hospitals, efforts to address this focus include many changes to clinical practice and provider workflow to decrease the risk of controlled substance diversion and reduce errors in controlled substance handling. Perioperative care areas have not been spared from these changes, with the management of controlled substances in the perioperative environment coming under increasing scrutiny. Regulatory groups such as The Joint Commission (Oakbrook Terrace, Illinois) and federal agencies such as the Drug Enforcement Administration (Springfield, Virginia) are reviewing how anesthesia providers manage the large amounts of controlled substances are administered to our patients. These entities are interested in ensuring that¹ :

1. Chain of custody of a controlled substance is maintained from the time it enters a hospital to the time it is either administered or disposed.

2. Documentation practices are accurate and auditable.

3. Controlled substance documentation in the anesthesia record matches documentation used to manage its dispensing and return.

4. Witnessing and wasting is performed according to hospital policy and tracked.

5. Discrepancies are reviewed systematically and integrated into diversion monitoring systems.

The lack of robust systems to allow providers to meticulously track controlled substances could increase the perception that anesthesia providers may be too casual in how they handle controlled substances, a perception supported by the high rate of controlled substance abuse by anesthesia providers.^2,3  Our institution experienced issues with both diversion as well as inadequate tracking of controlled substances. Commercially available automated anesthesia drug carts have software to manage controlled substances, but are currently not well integrated with anesthesia information systems resulting in published reports of controlled substance reconciliation error rates of greater than 5%.^4,5  These systems are unable to provide true real-time notifications on potential controlled substance management errors, which are essential for timely resolution of management issues. Although post hoc controlled substance dispensing surveillance systems have been described and evaluated in the literature, there are no data regarding accuracy (or quality) of point-of-care real-time perioperative controlled substance management tools.⁴  Our institution responded to inadequate controlled substance tracking by developing a new software application that addressed the entire perioperative controlled substance management process by integrating both pharmacy and anesthesia systems and enabling real-time error resolution.

We tested the hypothesis that implementation of such an application is associated with a reduction in management errors of an individual controlled substance medication or a package of multiple controlled substance medications used for one or more patients (sometimes known as a kit).

The authors followed the Enhancing the Quality and Transparency of Health Research Standards for Quality Improvement Reporting Excellence 2.0 guidelines in the development and structure of this manuscript.⁶ 

A notice of “not regulated status” (HUM00141108) was received from the Institutional Review Board (University of Michigan Medical School, Ann Arbor, Michigan) for analysis of data from a quality improvement project.⁷  The protocol for this quality improvement study was prespecified and presented at our departmental anesthesia clinical research committee before data extraction and after receiving Institutional Review Board determination of “not regulated status.”

The studied institution is a large, academic, tertiary care center in the United States. Approximately 450 anesthesia providers including residents, fellows, certified registered nurse anesthetists, and attending anesthesiologists, and an additional 30 pharmacy staff participate in more than 85,000 procedures per year, administering more than one million doses of controlled substances annually across six geographically distinct facilities (adult and pediatric inpatient facilities and several ambulatory surgery centers) and more than 100 anesthetizing locations. The management of controlled substances is a complex process in this setting. Historically, providers were supplied with a kit (defined above as a package of multiple controlled substance medications) for one or more patients that approximated what a provider would need throughout the day for multiple cases—a commonly used “kit per day” model with paper tracking and reconciliation. Ongoing review of controlled substance management practices, combined with unfortunate adverse events related to clinician abuse of substances (both controlled and uncontrolled), led first to change from a “kit per day” to a “kit per case” model still relying on paper tracking of the kits, and then an unsuccessful trial of automated drug dispensing carts in the operating rooms. This trial was deemed unsuccessful because of difficulties in maintaining chain of custody of controlled substances through provider breaks and reliefs and ineffective integration of real-time clinical documentation with drug inventory management.

To implement a more robust system for tracking controlled substances at sites requiring anesthesia care, our institution developed a web-based secure software application for controlled substance management in the perioperative area. This application manages the life cycle of the standardized controlled substance kits assembled by pharmacy staff and used by anesthesia providers, and includes data integration with electronic health record (pre-, intra-, and postoperative anesthesia documentation) and centralized operating room automated pharmacy dispensing and inventory tracking systems (Omnicell, USA).

In an attempt to minimize controlled substance management errors, a “kit-per-case” controlled substance management workflow was in place for one year before implementation of the software application; providers had already made the process and documentation changes necessary for the transition from “kit-per-day” to “kit-per-case” using a paper tracking system (appendix 1).

Our intervention was the implementation of the controlled substance management software application (appendix 2). The hospital’s controlled substance oversight committee approved development and implementation of a new perioperative controlled substance management system in March 2015. By September 2015, a pilot rollout was implemented at a free-standing ambulatory surgery center with health system–wide adoption at all anesthetizing locations by December 2015.

A web-based software application that integrated with our commercially available anesthesia information management system (General Electric Healthcare Centricity Anesthesia, USA) and pharmacy system (Omnicell, USA) was developed to track preparation, inventory and distribution, chain of custody, administration, and breakdown of controlled substance kits assigned to a specific case. An alerting module was developed within the application to notify users when workflow deviated from best controlled substance management practice. Examples of alerts include prompt notification to the anesthesia provider if a discrepancy is discovered, and timely escalation of discrepancies to departmental and hospital staff assigned to address such issues (such as a drug diversion compliance team).⁸  A reporting and discrepancy notification module was created for managers to review and follow up on any important controlled substance discrepancy in real-time. Fundamentally, the software application and workflow was designed to maximize the real-time recognition and alerting of variations from ideal processes rather than retrospective auditing after case completion. Overall cost for the implementation at our institution was approximately $500,000, and ongoing maintenance and support is about $50,000 per year across more than 100 anesthetizing sites.

After implementation of this controlled substance management application, a survey was sent to all anesthesia providers and pharmacy staff, asking them to compare the new system and process with the previous one.

The controlled substance management application is a web-based application built in C# with ASP.NET framework (Microsoft Corporation, USA) with interfaces between our anesthesia information management system and pharmacy management system. Anesthesia and pharmacy data are stored within a structured query language server database that is queried in real time (figs. 1 and 2).

Three distinct controlled substance workflow phases mirror the perioperative clinical workflow. Preoperatively, controlled substance kits are assembled on a routine basis and available for distribution to anesthesia providers in the immediate preoperative period. Pharmacy staff barcode scan the kit, and information about the kit (such as kit type and storage location) is stored within the application database. Controlled substance kits are specific by case type (General, Cardiac, Monitored Anesthesia Care, Obstetrics, and Pediatrics), and each kit type has specific quantities of controlled substances such as midazolam, fentanyl, morphine, ketamine, and ephedrine. Anesthesia providers obtain kits either from an operating room pharmacy or from an automated dispensing cabinet depending on location. In either scenario, the application logs this change in custody of the kit (from pharmacy to anesthesia provider), and transfers “ownership” to the anesthesia provider. The provider then assigns or “links” the kit to the anesthetic case the kit will be used for.

Intraoperatively, the application queries the anesthesia information management system for controlled substance administration and progressively decrements controlled substance totals requiring return to pharmacy, as these medications are documented in the anesthesia information management system. In-room provider (resident, fellow, or certified registered nurse anesthetist in our care setting) breaks and relief that occur during a case are noted in the application, and a transfer-of-care module within the application ensures that chain of custody is maintained from the leaving provider to the incoming provider. At each break or relief, clinicians transfer chain of custody of the controlled substance kit using a user ID and password verification step. To maximize usability, the user ID and password for the application is the same as the provider’s electronic health record credentials using single-sign authentication. Each handover identifies any discrepancies automatically by comparing anesthesia information management system and application calculations for used versus remaining controlled substances. If additional controlled substance is required in the middle of the case, then the central pharmacy or central automated drug cabinet dispenses additional vials or syringes of controlled substance to a provider (usually the attending anesthesiologist or other anesthesia provider not assigned to a room). This provider then gives the controlled substance to the in-room provider. Both interactions are documented with electronic attestation in the automated controlled substance management application.

Postoperatively, after care is handed over to the receiving provider (nurse in recovery room or intensive care unit), the provider will return the kit to the pharmacy, where both the anesthesia and pharmacy staff will jointly verify returned contents of the kit. After hours, when the operating room pharmacy may be closed, another licensed clinician will verify returned kit contents by visually inspecting the syringes and unused vials and electronically attesting in the application, and then the anesthesia provider places the kit in a secure lockbox for later pick-up by pharmacy staff. As per institutional policy, all unused medications must be returned to the in-hospital pharmacy or lockbox, whether in vial or syringe. (fig. 3). Leftover medication is not disposed of by the provider. A random sample of leftover medication is assayed by pharmacy staff and tracked in the application.

Throughout the period of anesthesia care, the application sends alerts via pages or application screen pop-ups to the anesthesia provider when required practice is not followed. Examples of these alerts include:

1. Failure of timely linking of a controlled substance kit to a case

2. Failure of timely transfer of kit between providers during a documented break or relief

3. Kit not returned to pharmacy within timely fashion after documentation of anesthesia end

4. Controlled substance documented in the anesthesia record not part of kit contents

The application also has a discrepancy notification module in which controlled substance quantitative errors detected by the application, anesthesia, or pharmacy staff are logged and notifications are sent via email to address the error, escalating to department and hospital compliance leaders if not addressed in a timely fashion. Routine analytical reports are generated for review by controlled substance management committees within the departments and hospital.

We conducted a retrospective before–after analysis of data for missing controlled substances from operating rooms. The preintervention period (consisting of paper tracking of kits) was from December 1, 2014 to August 31, 2015 (9 months). September 1, 2015 to May 31, 2016 (9 months) was the application implementation and user stabilization period across our institution. The study period was from June 1, 2016 to March 31, 2017 (also 9 months). A voluntary electronic survey (Qualtrics, USA) was administered in December 2016 to evaluate provider perceptions (fig. 5).

The primary outcome of this study was the rate of missing controlled substance medications, defined as any difference between the expected returned amounts of controlled substances versus actual returned amount. Secondary outcomes included rates for kits not returned to pharmacy and missing kit return witness signatures. All three of these events are tracked by U.S. Drug Enforcement Administration diversion program officers.

Summary statistics were displayed for both time periods as frequency of event and event rate per 1,000 cases with 95% CI. Additionally, the difference of event proportion between time periods with 95% CI was reported per 1,000 cases. Event rate per 1,000 was calculated as total number of events divided by the total number of cases to allow for consistency in denominator between periods of implementation. Outliers were assessed for all outcomes using box-and-whisker plots, and none were found. Outcome data were assessed for normality using histograms and were found to be normally distributed, therefore data is presented as mean ± SD. A two-sample independent t test for the equality of proportions with continuity correction was used to compare the paper kit discrepancy (preintervention) and application kit discrepancy (study period) event rates for the primary and secondary outcomes. A Joinpoint analysis was conducted to allow for potential nonlinear trend in the number of event counts across time,⁹  and the Bayesian Information Criteria was used to determine the best number of joinpoints to fit the model. Finally, means and SD, and slopes for trend in event counts were compared between the time periods using Student t tests and F tests, as appropriate. Analyzes were conducted using SAS v. 9.4 (SAS Institute, USA) and Joinpoint Regression software (Joinpoint Regression Program, Version 4.5.0.1, June 2017; Statistical Methodology and Applications Branch, Surveillance Research Program, National Cancer Institute, Bethesda, Maryland). Survey responses were evaluated between anesthesia providers (residents, fellows, certified registered nurse anesthetists, anesthesiology faculty) and pharmacy providers (pharmacy technician, pharmacist) using a chi-square or Fisher exact test, as appropriate. Survey responses were collapsed into a binary variable, with agree and strongly agree in one category and neither agree nor disagree, disagree, and strongly disagree in the other. Survey questions which directly compare the controlled substance management application to the paper-based system (3, 5, 6, and 7), were analyzed only between those providers who indicated they worked with both reporting systems.

All analyses were conducted using a two-sided hypothesis, and a P value less than or equal to 0.05 was considered statistically significant. Because this is a post hoc quality improvement study, no prospective power analysis was performed.

Ethical aspects of this implementation included the detailed review of provider administration habits and tracking of practice patterns that this new tool enabled. Our hospital’s and department’s compliance and Quality Assurance or Quality Improvement committees all agreed that benefits outweighed any potential risks, and that the implementation team would develop a process for feedback and modification of the tool as needed if these issues arose.

During the preintervention (paper tracking of kits) period 53,400 cases were examined, during the implementation period 57,670 cases were examined, and during the study (using the new application) period 65,911 cases were examined. In the preintervention versus study period analysis, the study period had significantly lower rates (per 1,000 cases) of missing controlled substance medications (0.42 vs. 1.12; difference 0.7, 95% CI: 0.38, 1.02, P < 0.001). For the secondary outcomes, we found that rates (per 1,000 cases) of kits not returned was also significantly lower (0.09 vs. 0.54; difference 0.45, 95% CI: 0.24, 0.66, P < 0.001). There was no statistically significant difference in the number of missing witness return signatures between the two kit tracking periods (0.17 controlled substance management application vs. 0.26 paper tracking; difference 0.09; 95% CI, −0.08 to 0.26, P = 0.350; table 1).

Joinpoint analysis exhibited different trends during the preintervention, implementation, and study periods. During the preintervention paper tracking period and early implementation period (December 1, 2014 to March 31, 2016), a statistically significant increase in the number of missing controlled substance medication errors was observed at a rate of 0.38 (95% CI, 0.10–0.66) more errors per month (P = 0.017, fig. 4A). Near the end of the implementation period, from March 2016 to June 2016, there was a nonstatistically significant decrease in the number of missing controlled substance medication errors, a rate of −2.52 (95% CI, −9.24 to 4.20) fewer errors per month (P = 0.471). During the study period, from June 2016 to the March 2017, there was no statistically significant change in the rate of missing controlled substance medication errors (P = 0.858). There was a significant mean decrease of −3.87 errors per month in the rate of missing controlled substance medication errors between the preintervention period (mean 6.67 ± 2.60) and the study period (mean 2.80 ± 2.04 errors per month, P = 0.002).

A significant decrease in the number of kit not returned errors at a rate of −0.10 fewer cases per month was consistent across the preintervention, implementation, and study periods (P = 0.032, fig. 4B). There was a significant mean decrease of −2.62 fewer cases per month in the rate of kit not returned errors between the preintervention paper kit tracking period (mean 3.22 ± 2.49) and the study period (mean 0.60 ± 0.84, P = 0.008).

There were no significant changes in the rate of missing witness signatures errors over the preintervention, implementation, and study periods (slope = −0.03; 95% CI, −0.09 to 0.03; P = 0.281; fig. 4C). Additionally, there was not a significant change in the mean errors per month of missing witness signature between the preintervention (mean 1.56 ± 1.01 and study periods (mean 1.10 ± 1.37, P = 0.160).

Responses were received from 189 of the 450 anesthesia providers (42%) and 17 of 35 pharmacy staff surveyed (49%, fig. 5). Of the anesthesia provider responses, 176 answered at least one question of interest. Eighty percent of those who completed the survey worked in both the preintervention paper kit period and the study period with the new controlled substance management application. Those who worked at other institutions (72 anesthesia providers and 3 pharmacy staff) said the application was better than or equal to the automated anesthesia dispensing machines and software at other institutions 48 of 75 (64%) of the time.

Those who worked in both the preintervention paper kit period and the study period with the new controlled substance management application (144 anesthesia providers and 14 pharmacy staff) largely agreed that using the new application was less time consuming and overall better than the paper kit system. When comparing anesthesia provider and pharmacy staff responses, significantly fewer anesthesia providers than pharmacy staff agreed or strongly agreed that the new application is less time consuming for controlled substance administration documentation than the paper-based system (108 of 144 [75%] of anesthesia providers vs. 13 of 14 [93%] of pharmacy staff, P = 0.003), and that overall the new application is better than the previous paper-based system (101 of 143 [71%] vs. 14 of 14 [100%], P = 0.012). The question asking whether overall the new application is better than the previous paper-based system was missing one response from the anesthesia providers.

Among all survey participants, significantly fewer anesthesia providers compared with pharmacy providers agreed or strongly agreed that the application’s witness waste functionality reduces the potential for controlled substance diversion (70 of 176 [40%] vs. 12 of 17 [71%], P = 0.019) and that the application helps to create a culture for the safe handling of controlled substances (108 of 176 [61%] vs. 17 of 17 [100.0%], P < 0.001).

We implemented an automated and integrated electronic controlled substance management application and workflow in response to increasing scrutiny of medication handling. We observed a statistically significant reduction in missing controlled substance medications and missing controlled substance kits. This statistical significance is also practically significant because regulatory and governance groups such as the Drug Enforcement Administration and hospital controlled substance committees require that we are able to accurately track and report on the life cycle of every controlled substance, and have a mechanism in place for discrepancy tracking, which we are now able to much more accurately. These types of software applications can play an important role in improving accounting of controlled substances through the perioperative period.

With the recent rise of opioid addiction and its devastating consequences moving to the forefront of public health challenges, regulatory groups (e.g., The Joint Commission) and enforcement agencies (e.g., Drug Enforcement Administration) are auditing hospitals to ensure that processes and documentation of controlled substances establishes a clear chain of custody, and that errors are captured and followed up in a swift and appropriate manner. Internal and external review at our institution revealed high error rates with paper kit tracking, including paper kit documentation mismatches with anesthesia record drug administration documentation, transfers of controlled substances unable to be tracked, and witness waste processes not completed correctly. Added up, these internal and external reviews determined that close to one-fifth of controlled substance kits with our old paper tracking system had at least one of these errors.

A study at another academic institution described implementation of a controlled substance electronic auditing tool that queried automated drug cabinet transactions and anesthetic information management system drug administration totals and presented to providers as a daily reconciliation report. Drug reconciliation errors decreased from 8.8% to 5.2% with deployment of their near-real-time tool.⁴  We observed a less than 1% rate of drug reconciliation errors using the controlled substance management application and process changes. Other efforts to manage controlled substance administration include deployment of automated anesthesia drug carts (for example, Pyxis Anesthesia Station or Omnicell Anesthesia Workstation). These devices are increasingly present in operating rooms and can allow for inventory management of drugs and other supplies. Although they also offer software to maintain chain of custody of controlled substances, this software is rarely integrated with anesthesia information management system data or equipped with alerting functionality and therefore unable to provide real-time notifications regarding controlled substance management errors.

An important part of our plan to improve our controlled substance management processes and address these findings was the development of our new software application. Consistent with new American Society of Health-System Pharmacists (Bethesda, Maryland) guidelines, software applications that integrate data, especially drug administration data, from anesthesia information management system and pharmacy systems, and enable anesthesia providers to maintain chain of custody of controlled substances in complex practice environments may reduce institutional and provider risk related to controlled substance management when combined with institutional culture change⁸ 

Our primary and secondary outcomes (number of missing controlled substance medications, missing kits, and missing witness return signature) were analyzed to determine whether error rate changed over the course of the study. Results from our analysis show a clinically significant change of error rates in missing medications due to the controlled substance management application. The error rate stabilized during the study period, once the application was widely deployed and well-integrated in provider workflow. The alerting functionality built into the system, as well as the heightened awareness of controlled substance accountability within our department, was conducive to a decrease in rates of missing medications between the preintervention paper kit period and the study period.

For kit return errors, joinpoint analysis demonstrated a decreasing linear trend of error rates through all three phases. This suggests that despite a statistically significant difference in kit return error rates between the preimplementation and study period, factors other than implementation of the new application may have led to the decrease in kit return errors. No significant change was found in the error rates of missing witness signatures between the preintervention period and the study period. Further review of data demonstrated that missing signatures largely occurred after hours, when kits were dropped in secure lockboxes for later review by pharmacy staff. Our opinion is that lack of direct pharmacy involvement in the return of controlled substances contributes to documentation errors, especially in after-hour situations where other clinicians able to witness a return (to ensure accuracy) may not be easily available.

At our institution, we learned that meticulous recordkeeping must exist to ensure that chain of custody of controlled substances is maintained from hospital arrival to administration or wasting for bolus and infusion medications. Each transfer of care in a case requires documentation of the transfer of care process for the controlled substances. Discrepancies between the anesthesia record and controlled substance management and reconciliation record must be identified, addressed, and rectified rapidly. Controlled substance management errors must be appropriately escalated to leadership, diversion control experts, and perhaps law enforcement, in a timely and predictable manner. Documented hospital policy must reflect these processes. Finally, if violations of these policies occur, thorough documentation and notification to regulatory bodies (e.g., Drug Enforcment Administration) may be necessary. Our experience is that this will be the minimum standard that sites must adhere to and will be judged against if a regulatory body audit occurs. There is little in the published literature regarding perioperative best practices and standards. In early 2019, the American Society of Health-System Pharmacists published guidelines regarding perioperative controlled substance management services that include recommendations similar to those above. These guidelines were still available online only and not accessible in the peer-reviewed literature as of April 2019.⁸ 

Results of the survey showed that both anesthesia providers and pharmacy staff perceived that the new application was more effective than both the previous paper tracking system and trial of the automated anesthesia drug carts. Pharmacy staff felt more strongly than anesthesia providers that the controlled substance management application was a better process for controlled substance handling, that controlled substance management was more time efficient with the new application, and that the application creates a culture of safe controlled substance handling. This is not surprising given that that pharmacy staff had primary responsibility for managing paper forms with the previous process. Overall, survey results indicated strong user acceptance of our application, but also room for improvement, especially with our witness waste process for anesthesia providers.

There are several limitations with this study. First, issues that are known to affect before–after implementation studies limited by historical controls may be present with this project as well.¹⁰  Benefit may be overestimated as we cannot quantitatively account for contemporaneous changes in culture and education regarding the importance of maintaining controlled substance chain of custody. Also, the discrepancy tracking process was far more cumbersome in the old, paper kit tracking process. However, this likely resulted in significant underreporting of discrepancies compared with the new software-based system. Our observations may be limited to specific processes and cultures at our organization and not generalizable to other organizations. Although there had been several Quality Improvement projects at our institution over the previous years focusing on controlled substance management, the unfortunate adverse events to hospital staff and Drug Enforcement Administration audit certainly were known to providers and may have played a role in the reduction of discrepancies. Heightened press attention to the controlled substance problem may also have led providers to be more vigilant with management of controlled substances. Finally, although this study demonstrates reduction in controlled substance management error, it is fair to state that these types of management and analytical tools, even in their finest form, may not prevent diversion at all. The typical practice pattern of our specialty enables providers who divert to develop systems that may bypass the checks developed by the most sophisticated of these surveillance tools. Further projects need to focus on how to best identify diverters while minimizing disruption to clinical workflow and provider privacy.

The automated controlled substance management application was reliably associated with a statistically significant drop in controlled substance management errors. These types of systems may ensure that appropriate accounting of controlled substances is performed throughout the perioperative period. They enable hospitals to comply with institutional policy and regulatory requirements. They may work synergistically with other programs to prevent diversion, but cannot be relied upon independently to prevent provider abuse of controlled substances. Our providers found the implementation of this application improved controlled substance chain of custody of as well as overall controlled substance management within our institution. Although the new application is unlikely to directly impact drug diversion, the survey results demonstrate that it is more user friendly and joinpoint analysis demonstrates it reduces errors handling controlled substances compared to paper processes. Future studies must evaluate replicability of our single-center observations and evaluate whether these systems actually prevent diversion. These studies could either use the described application which has modular design enabling integration with most anesthesia systems or other commercially available applications as they become available and develop similar functionality.

Support for this study was provided solely from institutional and/or departmental sources.

The authors declare no competing interests. The software evaluated in this study has not been patented or commercialized but is the property of the Regents of the University of Michigan, Ann Arbor, Michigan.