Hamilton Acute Pain Service Safety Study: Using Root Cause Analysis to Reduce the Incidence of Adverse Events

DURING the 1980s, the introduction of patient-controlled analgesia (PCA) and epidural analgesia combined with the desire to manage postoperative pain more effectively lead to the introduction of acute pain services (APSs).¹  These services are organized in a number of ways, but the common theme is to have a clinical team (that includes anesthesiologists and acute pain nurses) that round on patients with acute pain and manages their analgesia requirements, which includes writing and modifying the medication orders, and side-effect management. In most countries, PCA and epidural analgesia have become the foundation of postoperative analgesia.² 

Although effective at improving pain control, safety concerns have been raised regarding the analgesia modalities used on APSs. The opioids used in both PCA and epidural analgesia are sedating, with potential for progression to respiratory depression, respiratory and cardiac arrest, and even death. Severe hypotension is another complication of both PCA and epidural analgesia. Beyond the opioid side effects, epidural analgesia can also result in prolonged or progressive motor block and rarely spinal hematoma or spinal abscess. The incidence of these complications has been reported to be relatively low, approximately 1% or less and varies depending on the precise definition used and the analgesia modality.^3–6 

The three APSs (McMaster University Medical Centre, Henderson General, and Hamilton General, Hamilton, Ontario, Canada) at Hamilton Health Sciences (HHS) began an acute pain database project for the purpose of clinical documentation, quality assurance, and research in February 2002.^* Until 2006, adverse events (AEs) were discussed informally during the monthly HHS APS meetings in an effort to address safety issues. In this forum, it was seen that many of the safety issues and potential solutions raised were not further investigated or followed-up. Consequently, the AEs persisted, and similar issues were raised from meeting to meeting.

The root cause analysis (RCA) approach, advocated by the U.S. Veteran Affairs National Center for Patient Safety (Ann Arbor, Michigan), is a process to deal with AEs from a systems level.^†7  This process is not an accountability or blaming system but rather a learning system with the focus on determining: (1) what happened, (2) why it happened, and (3) what is necessary to prevent similar incidents from happening in the future. The Canadian Patient Safety Institute (Ottawa, Ontario, Canada) was established in 2003 with the goal of collaborating with health professionals and organizations to build a safer healthcare system. The Canadian Patient Safety Institute developed a framework for RCA which was adapted from the National Center for Patient Safety approach.^‡

Although the RCA process seemed a promising and a logical solution to our persistent problem with AEs on the APS, there was insufficient evidence to demonstrate that it would be effective. The Hamilton Acute Pain Safety Study was a prospective before–after cohort study that evaluated the effectiveness of formal RCA on the incidence of AEs amongst patients on an APS.

After research ethics approval (by the HHS Research Ethics Board, Hamilton, Ontario, Canada), this study was conducted at three tertiary care hospitals in Hamilton, Ontario, Canada. The study sites included: McMaster University Medical Centre, Hamilton General Hospital, and the Henderson General Hospital. The study was designed as a prospective cohort study that compared the incidence of AEs (table 1) before and after the introduction of a formal RCA process.

The “before” cohort included all patients enrolled into the APS from February 2002 to July 2007 (before the introduction of a formal RCA process), the “study cohort” included all patients enrolled into the APS from August 2007 to December 2008 (during the active formal RCA process), and the “after” cohort included all patients enrolled from January 2009 to December 2009 (after the RCAs were complete and the recommendations were sent out). The before cohort of patients may have low estimates of AEs as this is a group of historical controls where data were collected before the implementation of the RCA intervention.

At the request of the investigative team, the Chief Executive Officer of HHS circulated a memorandum throughout the hospital administration describing the study and requesting the staff to do their best to respond to recommendations arising from the study. The study team (and 40 other hospital staff) underwent a full-day workshop on the RCA process which was run by the Canadian Patient Safety Institute. In addition, a member of the study team (Dr. Musson) with a background in medicine and social psychology (with specific experience in human factors and team performance analysis) attended the first three RCAs to guide the team in focusing on the system as opposed to just on the medicine and physiology when attributing root causes to AEs.

During the study period, 112 events occurred and 10 of them, which were considered severe and likely to recur, were flagged for RCA. The study team followed the Canadian Patient Safety Institute RCA Framework to identify root causes for these events and to develop action plans to prevent them from recurring, table 2. An online AE reporting system was developed to record the results of these investigations and to manage the follow-up of the resulting recommendations.

The primary outcome for the study is the difference in AE rates before and after the implementation of a formal RCA process. Secondary outcomes included the root causes identified for each AE type, the number of recommendations generated, and the number of recommendations completed.

The pre- and post-RCA AE rates (clinical outcome) were categorized by analgesia modality (PCA or epidural) and described by count (percent). The recommendation status designations were also summarized by count (percent). We compared pre- and post-RCA rates using a Mantel–Haenszel chi-square test, and Fisher exact test was used when counts were less than five. Severe pain events were not included in the pre-post analysis as this was a measure of quality rather than safety, and the high number of these events would have dominated the results overall. The level of significance was set at α = 0·05. We used the Bonferroni method to adjust the overall level of significance for multiple comparisons. Specifically, we divided the α by the number of comparisons presented on each table. These analyses were performed using STATA 10·1 (College Station, TX).

During the entire study period, from February 2002 to December 2009, a total of 35,384 patients were tracked on the three APSs within HHS. There were 866 AEs of some type or one AE for every 41 APS patients. The flow of patients through the study is illustrated in figure 1, and major milestones for the study are summarized in appendix 1.

There were 23,198 patients in the before RCA cohort with 658 AEs (or 1 in 35 patients).

Ten of these AE were flagged for formal RCA: one case of cardiac arrest, three cases of severe respiratory depression, one case of severe hypotension, one case of an unresponsive patient, one case of delirium, one case of uncontrolled severe pain, one case of inappropriate anticoagulation, and one case of prolonged or high motor block.

There were 4,352 patients in the after RCA intervention follow-up cohort with 96 AEs (or 1 in 45 patients). Overall, the combined AE rate in the after RCA intervention group (1.47%) was significantly less than in the before RCA group (2·35%), P value less than 0·001.

A total of 26 unique root causes (appendix 2) were identified from 10 RCAs and a recommendation was formulated for each one. For each RCA, the number of root causes (and associated recommendations) identified varied from 3 to 10; the mean was 7·5 per case. The 26 recommendations were applied a total of 76 times, and each one was used a mean of 2.9 times amongst the 10 RCAs. The most common recommendations included: use visual prompts on PCA and epidural pumps to remind staff of the pain monitoring protocols, purchase more portable monitors to facilitate vital sign monitoring, develop a back-to-basics nursing campaign, streamline nursing documentation on an electronic documentation system, and develop an annual e-learning pain management update for nursing staff. The root cause and recommendation types were policy/procedure (41%), environment/equipment (30%), training (17%), fatigue/scheduling (11%), and barrier (1%). Of the 26 action types for the recommendations, one was “eliminate” and the remaining 25 were to “control.” The status of these 26 recommendations at the end of the study was 23 (88.5%) were “completed,” one (3·8%) was “to be completed,” and two (7·7%) were “not to be completed.” The two recommendations not completed included: (1) purchase of a new call bell system in one of the intensive care units to replace a defective unit (not completed because this intensive care unit was closed and moved to a new facility) and (2) develop a formal intensive care unit discharge criteria outlining parameters necessary for discharge (not completed because the intensive care unit staff felt this was not necessary as discharge criteria were well described in the literature).

Although eight different types of AEs were assessed using the RCA process, some common themes emerged. In the 10 AEs that were analyzed, most (80%) had significant gaps in recording of vital signs assessments (table 3). Insufficient knowledge of pain management protocols and analgesia principles was a common finding and was identified as a root cause in 80% of the RCAs. Similarly, inadequate ongoing staff pain education (after the initial staff orientation) was noted in the majority (80%) of the cases. Equipment availability issues were also important, with most (80%) of the RCAs reporting insufficient portable monitors on the wards. Nurse staffing issues (staff numbers and patient assignments) were identified in five of the RCAs, but only two showed inadequate staffing at the time of the AEs.

The pre- and post-RCA AE rates are summarized in table 4. The most common AE identified was uncontrolled severe pain (7.3%), and this was followed by severe hypotension (1.1%), respiratory depression (0.6%), inappropriate anticoagulation (0.3%), pain pump programming error (0.08%), prolonged motor block (0.05%), epidural abscess (0.04%), severe sedation/unresponsive (0.03%), epidural hematoma (0.02%), cardiac arrest (0.02%), delirium (0.01%), and death (0.01%).

The rates of severe uncontrolled pain, respiratory depression, and pain pump programming errors were significantly greater for PCA opioid patients when a continuous infusion was used in comparison with patients treated with PCA boluses alone (P < 0.001, <0.001, and <0.001, respectively) or with epidural patients (P < 0.001, <0.001, and <0.001, respectively). The rate of respiratory depression was 2.4 times greater for PCA plus infusion patients in comparison with PCA bolus only patients and three times greater in comparison with epidural patients.

The rate of severe hypotension was significantly greater for epidural patients in comparison with PCA opioid bolus only patients (P < 0.001) or PCA plus infusion patients (P < 0.001). Specifically, the hypotension rate for epidural patients was six times greater than the rate for PCA bolus patients and four times greater than the rate for PCA plus infusion patients.

This 7-yr acute pain safety study at three tertiary care hospitals resulted in information on 866 AEs amongst 35,384 patients. Formal RCA (over a period of 15 months) of 10 of these AEs resulted in 26 unique recommendations, of which 24 were either completed or in the process of being completed after 1 yr of follow-up. The resulting impact on patient safety (amongst a cohort of 4,352 APS patients) was that the overall AE rate was reduced from 1 event per 42 APS patients to 1 in 68. RCA was associated with reducing the incidence of respiratory depression and severe hypotension, but not severe pain, cardiac arrest, severe sedation, delirium, inappropriate anticoagulation, epidural abscess or hematoma, or death. The incidence of pump programming errors was also reduced after the RCA intervention, but this AE was not addressed during the study (as one did not occur during the intervention or follow-up period).⁸  The most common root causes that were identified during the RCA process were insufficient vital sign monitoring, inadequate pain education updates for ward nurses, and difficulty accessing portable monitors on the wards. With regard to the inadequate vital sign monitoring, 8 of the 10 cases had significant gaps between vital signs assessments, and some of these gaps were as long as 13 h.

The most common events identified were severe pain (1 in 14), severe hypotension (1 in 90), respiratory depression (1 in 170), inappropriate anticoagulation of epidural patients (1 in 360), and pump programming errors (1 in 1,250). It is not exactly clear why the incidence of severe pain went up after the RCA intervention, but it may have been related to the move to reduce the dose of PCA opioids. This increase in severe pain underscores the reality that safety interventions may result in some unintended harm, and that the benefits they may bring could have a cost. Patients treated with PCA opioids plus a continuous infusion of IV opioids had a significantly greater incidence of severe uncontrolled pain (4 times), respiratory depression (2.4 times), severe hypotension (1.3 times greater than PCA bolus patients), and pump programming errors (12 times). The increase in severe pain with continuous infusions likely reflects the practice whereby patients having surgeries with known pain management challenges are more likely to be treated with a continuous infusion. Hypotension (not surprisingly) occurred significantly more frequently amongst patients treated with epidurals. The incidences of rare AEs were: death (1 in 8,800), epidural hematoma (1 in 5,400), epidural abscess (1 in 2,700), and cardiac arrest (1 in 5,000).

The strengths of this study are that it included a very large cohort of patients, used a comprehensive and thorough RCA process (based on the U.S. Veteran Affair’s National Center for Patient Safety and Canadian Patient Safety Institute RCA frameworks), included participation of hospital leadership, and implemented an online reporting system to ensure that recommendations were not “lost to follow-up.” It also included three different hospital sites, representing a wide scope of clinical services and encompassing both pediatric and adult patients.

It is challenging to distinguish the direct impact of the RCA process on AE reduction in this study as opposed to the more general effect of a positive impact on performance (in this case safety) simply because people knew they were being studied (an effect referred to as the Hawthorne effect in studies of workplace behavior).⁹  Indeed, the notably lower rate of AEs during the RCA phase of the study may support this conclusion. In our opinion, there is a direct effect of the RCA process: the study team has been under study conditions since 2002 and actively tracking AE. The addition of the RCA process was an extension to an existing monthly quality assurance safety meeting, which we believe supports a direct impact of the RCA process as the main driver of the observed changes. More importantly, the AE rate remains low (1.47%) 3 yr after the last RCA case was completed, suggesting a persistent system and behavior change.

The before–after design of our study did not control for temporal changes to the system apart from the RCA intervention. Hence, it is possible that some of the changes in the event rates could be attributed to factors (e.g., improved education amongst staff, increased focus on safety by hospital administration, better management of patient’s comorbidities, and so on) other than the study intervention. Another weakness of our study is that some outcomes may have been underestimated. Our estimate of postoperative delirium was very low (0.01%), much lower than other estimates of postoperative cognitive dysfunction that have been as high as 41% in older patient groups.¹⁰  This difference is likely due to the fact that our APS team was trained and focused on capturing the main analgesia outcomes (pain scores, side effects, and the more common AEs: death, respiratory depression, hypotension, medication errors, pump programming errors, epidural site infection, and motor block) and only picked up clinically obvious cases of acute cognitive dysfunction.

Despite the challenges in organizing and managing the RCA process and its resulting recommendations, the RCA approach offers numerous advantages over the current alternatives (such as discussing AEs in a more informal manner). Most importantly, RCA encourages staff to address system issues and the resulting recommendations are given much more credibility through the endorsement by senior administration, which in turn increases the chances of the recommendations being implemented. Some of the biggest obstacles in conducting RCAs were getting all the key stakeholders together for the necessary meetings, identifying the most responsible person to follow up with specific recommendations, getting buy-in for recommendations, and getting safety recommendations completed when there were significant costs involved. For example, the study team’s recommendation for a wireless respiratory monitoring system involves a capital investment of hundreds of thousands of dollars upfront and tens of thousands of dollars annually to maintain such a system. Based on our experience with this process in the three hospitals under study, we summarize the advantages and disadvantages of the RCA technique in table 5.

Other studies have shown benefits from the RCA process. A 2005 retrospective study involving 100 Veteran Affairs acute and long-term care facilities, which used RCA to examine the incidence of falls and related injuries, demonstrated that 176 RCAs resulted in 745 actions, and 61% of these were completed.¹¹  Subsequently, 34% of the facilities reported a reduction in falls and 39% reported a reduction in major injuries from falls. A 2008 narrative review article by Percarpio et al.¹²  assessed the evidence for RCA in published literature. This study found that RCA emerged in the literature in the late 1990s and since then 11 case studies have been published that measure RCA effectiveness, 3 using clinical outcome measures (before/after RCA AE incidence rate comparisons) and 8 using process measures (percent of actions implemented). All 11 studies reported an improvement in safety with the RCA process. Our current APS RCA study is an addition to the small number of studies that have prospectively assessed the effectiveness of the RCA process by measuring both clinical and process outcomes. Given that our study showed safety improvement in three different institutions with different surgical services (general, orthopedic, gynecology, vascular, urology, and plastics) and patient populations (pediatric and adults), it is possible the benefits of the RCA process we observed could be realized in other hospitals.

In conclusion, our study found that after the introduction of a formal RCA process with comprehensive follow-up of the recommendations, there was an improvement in the overall safety of the APS, and specifically a reduction in the incidence of respiratory depression, severe hypotension, and pain pump programming errors. The incidence of serious AE amongst APS patients is low but remains a concern because this can be viewed as an iatrogenic illness that can affect all patients. The next research steps should include a system-level RCA study where participating hospitals are randomized to a formal RCA process or standard care, and the study budget should include funding to assist in implementing the recommendations.