“Nonroutine Events” as a Nonroutine Outcome for Perioperative Systems Research

Perioperative safety and systems research has long been stymied by the weak, inconsistent relationship between healthcare interventions and classic outcomes of interest such as mortality. As members of the anesthesia care team, we know intuitively that our individual and collective behaviors influence patient outcomes. Why, then, is it so difficult to empirically demonstrate such a relationship in systems research? Outcomes such as mortality in clinical and health services research are rare, limiting statistical power to show associations. In this issue of Anesthesiology, Liberman et al.¹  present an alternative to rare outcome measures—one that they call “nonroutine events.”

In creating and evaluating systems meant to reduce harm, industrial engineers consider a principle represented using the “accident triangle” (fig. 1) that relates frequent, low-importance events to infrequent, high-importance events such as mortality.²  Quality and safety professionals in health care have largely adopted this principle, creating robust systems to capture near misses and deviations from care processes. Taking cues from aerospace, aviation, manufacturing, and nuclear power, healthcare safety systems such as critical incident reporting systems examine threats both prospectively and retrospectively and strongly emphasize voluntary reporting of events and near-events by on the ground staff. Such voluntary reporting systems are valuable because they enable the identification of events with the potential to lead to harm and, across reports, enable the characterization of patterns that might signal a systemic problem. Unfortunately, even with mechanisms to protect reporters from retaliation or even self-incrimination, reporting systems only capture what people choose to report. For this reason, there is no information about base rates, which limits the ability of these systems to shed light on safety trends over time.

An alternative approach to voluntary reporting for characterizing near misses is to quantify the number of events across a group of cases. In their research article, Liberman et al. investigate nonroutine events by examining more than 500 anesthesia cases through collecting audiovisual recordings and provider surveys about the incidence of nonroutine events.¹  In their study, the authors define nonroutine events as “any aspect of clinical care perceived by clinicians or observers as a deviation from optimal care for that patient in that clinical situation” (emphasis in original); this definition was first proposed by Weinger et al. in 2003.³  The definition of nonroutine events purposefully includes subjective judgment to include many different types of events at the base of the accident pyramid that may lead to outright adverse events. The authors found that nonroutine events were present in more than 20% of operative cases, with more than a third (7.6% of the total) having more than one nonroutine event. The nonroutine events included both near misses (79%) and events resulting in patient injuries (21%). Cases with nonroutine events were more likely to involve older patients, patients with American Society of Anesthesiologists Physical Status III or IV, and junior anesthesia residents (as compared with senior anesthesia residents or nurse anesthetists). No association was found between measures of workload and vigilance and the presence of nonroutine events.

A commonly leveled criticism is that efforts to improve quality and safety focus on process outcomes such as nonroutine events to the exclusion of patient outcomes. Some argue that process outcomes can be “gamed,” drawing concerns about their validity. Perhaps for these reasons, in health services research, we have seen a shift away from the acceptance of process outcomes as valid measures of intervention effectiveness. This approach privileges big ticket outcomes such as mortality or organ failure. Although this focus on patient outcomes is well-intentioned, this limits the apparent impact of quality and safety-focused interventions. Most patient outcomes are multifactorial; as an example, inpatient mortality could be influenced by preexisting systemic disease, the complexity of a given surgical procedure, and nutritional status. No quality- or safety-focused intervention is likely to overwhelm the influence that these factors have on mortality. The multifactorial nature of outcomes like mortality contribute to the improbability of achieving adequate power to demonstrate an impact, yet we hold quality and safety studies to the same standards expected of large, multisite trials.

Nonroutine events offer a promising middle ground. As demonstrated by Liberman et al., these events are quite common and are associated with more serious events that the authors call “patient impact events,” which included difficulty with tracheal intubation, unstable hemodynamics, and equipment problems, among others.¹  It is plausible that a decrease in nonroutine events would translate into a decrease in patient impact events, which in turn would translate into a decrease in outcomes usually grouped into the category of morbidity and mortality. For small, single-center studies, it might be possible to select both a study design and sample size able to demonstrate a difference in nonroutine events, whereas larger, multicenter studies could more precisely describe the relationship between nonroutine events, patient impact events, morbidity, and mortality.

As with any outcome, however, nonroutine events have limitations. First, nonroutine events are not routinely collected elements of care, requiring that prospective data collection be undertaken to quantify them. If this work is done in a rigorous fashion as was done by Liberman et al., it could be costly and time-consuming to conduct observations and/or audiovisual recording and to recruit clinicians to report nonroutine events. Audiovisual recording is itself fraught, as institutional policies and local, regional, and national legislation present impediments to the routine recording of clinical care. Second, nonroutine event measurement is subject to selection bias. Collecting nonroutine events from all cases in a given clinical area over a prespecified period of time might attenuate this bias, but differential weighting might still be needed to estimate the incidence of nonroutine events across a given population. Third, nonroutine events need to be classified by expert raters; such classification introduces another source of bias. In the Liberman et al. study, raters were trained and measurements of both intrarater and interrater reliability were obtained. For broader applications of nonroutine as an outcome, similarly rigorous training and quality assurance procedures would be needed. Fourth, despite prospective collection, nonroutine events are likely to be subject to reporting bias. As has been learned from the experience of the United States National Aeronautics and Space Administration Aviation Safety Reporting System, fear of judgment, retaliation, or incrimination must be addressed to facilitate reporting.⁴ 

Clearly, patient outcome improvement is the goal of improvement research. Care must be taken not to conflate process or intermediate outcomes with patient outcomes; the former are valueless without the latter. Larger, ideally multicenter studies are needed to validate nonroutine events to test the relationships between quality, safety, implementation, and patient outcomes. If these limitations of nonroutine events can be addressed or contained, this outcome bears promise as a leading indicator of harm and untoward outcomes for our patients. In the short term, researchers could consider using nonroutine events as a means to test the effects of a safety or quality intervention. In addition, those in positions related to results dissemination (e.g., journal reviewers and editors) should give consideration to supporting papers with nonroutine events as primary or secondary outcomes. As with research endeavors such as the Standardised Endpoints in Perioperative medicine initiative,⁵  it is important to promote the use of a common set of outcomes—outcomes with a plausible connection to the exposure of interest—when conducting prospective research. In so doing, we set the stage for combining data and gaining new insights into patient care and outcomes.