A Facilitated Survey Instrument Captures Significantly More Anesthesia Events Than Does Traditional Voluntary Event Reporting

After obtaining institutional review board approval, a trained nonclinician research assistant interviewed anesthesia providers using the CONES structured survey tool on 87 days, selected at convenience for the researchers, over a 30-month period at the Veterans Affairs San Diego Healthcare System, San Diego, California. Because the survey was specifically targeted to capture NREs occurring in the operating rooms, the assistant interviewed anesthesia providers after they had handed off their patients to the postanesthesia care unit nurse or, if necessary, during slow periods of the next case. A standardized interview form was used to document NREs based on the clinicians’ response to nine open-ended questions ( appendix 1). After identifying an NRE, additional questions sought data about its etiology and potential contributor factors.12Only the data from the CONES interview were available for subsequent analysis and event coding.

Data collection rarely took more than 5 min. No identifiable patient or provider data were obtained. The NRE interviewers were independent of the department, and the process was strictly confidential and anonymous. Interviewers were college educated but had no clinical experience before their comparable training in preparation for this study. The training began with a month of basic reading about anesthesia and observation of providers in the operating room, similar to a medical student rotation. Trainees observed both real and videotaped cases under close supervision of more experienced observers. Interviewers were subsequently trained in the CONES interview process by shadowing an experienced interviewer during postanesthesia care unit interviews over multiple days of data collection. Expert reviewers were not actively providing patient care at this institution during this study (except for the principal investigator, who only served as a final arbiter). The two primary reviewers did not collect data, interact with the data collectors, or attend departmental QA conferences.

The intent of the CONES tool was to capture all events that were thought to deviate from optimal care. The CONES was designed to first ask general questions in an unbiased manner about anything that might have deviated from routine or optimal for that provider ( appendix 1). Therefore, the initial nine questions were used to identify NREs. The cause of an NRE is usually multifactorial, and therefore, the questions were intended to identify a variety of these factors. The initial question sequence was the same every time, and an affirmative response to any of the first nine questions led to the second set of questions, several of which were open-ended and required more detailed responses, designed to ascertain the nature and etiology of the putative NRE. The CONES questions were refined through an iterative process based on pilot postanesthesia care unit surveys as well as feedback from clinician participants, interviewers, and reviewers. All of the items in the final CONES version were found to elicit meaningful responses.

Over the same period, the Veterans Affairs San Diego Healthcare System’s QA system collected data regarding specific anesthesia care events. After every anesthetic, the anesthesia service expected its providers to fill out a form indicating whether one or more categories of adverse events had occurred during the case ( appendix 2). If a QA form (which allowed the option of checking a “no event” box) was not completed for a specific anesthetic, the department administrator contacted the responsible certified registered nurse anesthetist or anesthesia resident and asked him or her to complete the form for that case. QA data forms and associated peer-review analyses were deidentified and transferred to the study database. Only data about the patient and situation that appeared in the final QA report were included in the analysis.

For surgical population demographics, data sets of all procedures performed in the operating room during the study period were acquired from both the anesthesia and surgical services. The databases were compared and found to be largely congruent. The anesthesia database was thought to contain more accurate data about patient age, anesthetic type, and American Society of Anesthesiologists (ASA) physical status. The surgery database was used to provide the type of surgery performed.

All identifiable patient and provider information was removed from all data sets. Each case was assigned a unique randomly generated case number, as was each NRE identified. Data obtained from the two methods, along with any demographic, medical, and peer-review (for QA data only) summaries were entered into a custom password-protected Filemaker Pro® (Santa Clara, CA) database. NRE data were entered into the database in a uniform manner, regardless of the source (i.e. , whether CONES or QA derived). Because provider identity was excised, each case was treated independently even though the same anesthesia provider or surgeon might have been involved in several cases. Because it was possible for the two methods to both capture the same operative event, it was prospectively decided that, to address this potential statistical confound, all duplicate cases would be excluded from data analysis (table 2).

Two clinical experts independently reviewed each event in both data sets and determined whether reported events met the definition of an NRE. For all of the analyses described below, if there was disagreement between expert reviewers, a consensus was reached after discussion of each disputed case. If agreement was not obtained, a third expert reviewed the case, and the majority opinion was selected. To reduce hindsight bias, reviewers were shielded from knowledge of NRE outcome or clinical course until the end of their review. Reviewer-completed ratings and annotations were stored in the database. Before any statistical analyses, all events were reviewed one last time to ensure maximal consistency of classification and coding of all cases.

Each NRE identified from QA- and CONES-derived data were categorized according to its patient impact (PIE), patient injury type, outcome severity, and putative contributing factors (see definitions in table 1). Patient injury was defined as any unanticipated side effect or complication of anesthesia that affected the patient’s postoperative course or quality of life. Therefore, with this patient-centric definition, “injuries” included events that required the need for a higher level or prolonged postoperative care (e.g. , surgical intensive care unit admission instead of discharge home) or emotional distress (e.g. , case cancellation, intraoperative recall).

A structured taxonomy of 337 event descriptors, organized hierarchically by clinical manifestation, was applied to each NRE. This comprehensive taxonomy described both patient impact and injury events within major categories as well as subcodes for specific events. For example, under the major category of Airway Events, one would find codes for difficult ventilation, difficult intubation, esophageal intubation, inadvertent endotracheal extubation, and so forth.

Over a 1-yr period before this study, multiple anesthesia experts in our department refined this taxonomy iteratively. In collaboration, Dr. Ping-Wing Lui et al.  15at the Chang-Gung Memorial Hospital, Taoyuan, Taiwan, successfully used a version of this taxonomy in a quality improvement study involving almost 200,000 anesthetized patients. The event classification scheme was iteratively developed and refined before being applied in this study.12 

To simplify analysis and interpretation, the discrete PIE categories were condensed into eight composite categories (table 3) based on common organ systems affected (e.g. , all primary pulmonary events were grouped together) or presumed mechanisms of action (e.g. , the Anesthesia category included needle-related complications of regional anesthesia and of central venous catheterization).

For each patient injury, the reviewer used the ASA Closed Claims Study Injury Severity Scale to code the magnitude of injury.16Injuries were rated from 0 (no injury) to 9 (death) depending on the severity of patient harm. Emotional injury was rated as 1. Insignificant, minor, and major temporary injuries were rated as 2, 3, and 4, respectively. Minor, significant, major, and grave permanent injuries were rated as 5, 6, 7, and 8, respectively. Because it was possible for each case to have several NREs and for each NRE to have multiple outcomes, the most severe outcome for all NREs in a case was assigned to that case.

Finally, each reviewer assessed whether any of 21 discrete contributory factors (see table 4footnote) seemed to play a role in the occurrence of each NRE. This postevent analysis was used to delineate differences between types of NREs and potential common sources or mechanisms of event etiology deserving of intervention. The contributory factor classification scheme was established at the beginning of the project based on the literature and 20 yr of human factors experience. For analysis, the contributory factors were condensed into 11 composite contributory factors (table 4).

The principal objective of this study was to evaluate differences between CONES and traditional QA data in terms of case variables (i.e. , patient, anesthesia, and surgical factors), NRE incidence, presence of patient impact or injury, outcome severity, and putative contributory factors. Case variables, NRE incidence, and presence of patient impact or injury were evaluated by case using the Pearson chi-square test with the Yates continuity or the Fisher exact test. Patient outcome severity score data were categorically grouped (0, 1–3, 4, 5, >6) and then analyzed using the Fisher exact test.

The 11 composite contributory factor categories were considered statistically independent, because dependencies were accounted for when collapsing the original discrete categories into the composite categories. A logistic regression model was used to test for differences between CONES and QA with respect to the presence of individual contributory factors. Individual factors were then tested using the Pearson chi-square test with the Bonferroni correction applied to adjust for testing multiple comparisons.

Similarly, PIEs were collapsed into eight categories that were considered independently. A logistic regression model was used to test for differences between CONES and QA with respect to the count distribution of individual PIEs (0, 1, or > 1). The Pearson chi-square test was used to compare the distribution of PIEs between CONES and QA. Individual categories were then tested using the Pearson chi-square test with the Bonferroni correction applied to adjust for testing multiple comparisons.

All statistical analyses were conducted using the R statistical software package, which is an on-line open-source software application.††Differences were considered statistically significant if P  was less than 0.05. P  values less than 0.001 were reported simply as less than 0.001. Samples with counts of 3 or fewer did not have P  values computed. Multiple comparison adjustment via  Bonferroni correction was applied when required.

The active survey method using the nonroutine event construct identified multiple events that may have otherwise gone undetected and therefore unexamined with traditional QA reporting. During the 30-month study period, a total of 8,303 anesthetic associated procedures were performed at this hospital. For at least several hours on 87 weekdays over a period of approximately 630 regular workdays, 183 CONES surveys were performed, representing 2.2% (183 of 8,303) of cases. Of this sample, 31.3% of the cases contained an NRE. During the same period, a total of 1.9% (159 of 8,303) of all the cases were reported and evaluated per traditional QA analysis according to departmental policy. Two cases captured by both methods were excluded from subsequent analysis (table 2), and all data reported hereafter reflect this.

Patient and provider variables in the CONES data were highly reflective of the surgical population (table 5). By contrast, patients in the QA data set were slightly older (P < 0.05). A greater proportion of QA cases were performed by residents (P < 0.001). General anesthesia was more common, and monitored anesthetic care was much less frequent in the QA data set compared with the surgical population as a whole (P < 0.001). The QA data set contained patients with a higher ASA classification (i.e. , sicker patients; P < 0.001), and emergent cases were more common (P < 0.05).

Compared with the population, cardiothoracic (P < 0.05) and ophthalmologic (P < 0.01) surgical procedures were underrepresented in the CONES data, whereas plastic surgical cases were overrepresented (P < 0.001). Neurosurgical (P < 0.001), cardiothoracic (P < 0.001), and vascular (P < 0.01) surgery cases were overrepresented in the QA data. The QA process did not detect any ophthalmology case events.

Nonroutine events occurred in 55 (30.4%) of 181 CONES cases and 156 (98.1%) of 159 QA cases (P < 0.001). Among cases with NREs, there was patient impact in 74.5% versus  96.2% of CONES and QA cases, respectively (P < 0.001). Similarly, patient injury among NRE cases occurred in 23.6% of CONES and 60.3% of QA cases (P < 0.001). Severity of injury was significantly worse in the QA data (P < 0.001; fig. 1). However, if one linearly extrapolates the incidence determined from the CONES data sample to all surgical cases during the study period, the CONES method detected a sevenfold greater incidence of patient injury events (7.7% for CONES vs.  1.1% for QA). The greater frequency of event identification using the CONES process suggests that many more safety-critical events, including near misses, could be captured in a robust QA system that incorporated additional methods of event identification.

Among CONES cases, the most common PIEs were airway, cardiovascular, other (miscellaneous), and surgical events (table 3). Among QA cases, cardiovascular, pulmonary, airway, and patient disposition issues were most frequent. In the logistic regression model, PIEs related to the neurologic, cardiovascular, and pulmonary systems were less frequent in the CONES data set compared with the QA data (all P < 0.001). A lower proportion of CONES NRE-affected patients were sent to higher levels of postoperative care (e.g. , fewer required care in the intensive care unit). However, this difference did not attain significance in the overall logistic regression.

Among CONES cases, the most common factors seeming to contribute to NRE were Patient, Provider,‡‡Equipment, and Preoperative preparation factors. For QA-derived NRE, Patient, Provider,‡‡Surgical, and Drug-related factors were most frequent (table 4). Patient-related factors (i.e. , preexisting disease or unexpected response) were less  commonly cited in CONES NREs (52.7%) than those detected with the QA process (84.6%; P < 0.001). In contrast, equipment factors (especially poor usability) (27.3% vs.  4.5%; P < 0.001) and environmental factors (10.9% vs.  0.6%; P = 0.001) were more commonly cited in CONES compared with QA NREs. In the logistic regression model, only surgical (P < 0.05) and patient (P < 0.005) factors proved significantly different between the two data sets.

There were patterns to CONES-detected NREs that identified patient safety issues that might not have emerged through traditional QA reporting. For example, a number of cases were reported in which significant desaturation occurred after extubation. This finding led to further study, analysis, and implementation of simulation-based training and more explicit department-wide appreciation of extubation risks.

This study examined the potential value of the NRE-based survey technique as a complement to traditional QA. Applying CONES to a representative sample of surgical patients revealed an incidence of patient injury seven times greater than that suggested by data captured using a traditional QA process. The CONES approach revealed patient impact events (i.e. , events associated with changes in patient physiologic or psychological status) 12 times more frequently than self-reported QA data. The QA process generally captured the more serious patient injury events, but it did not detect many mild to moderate injury events and most near misses. CONES captured a broader array of the range and magnitude of patient safety issues in perioperative care than the traditional method of event detection. Its greater sensitivity, provider compliance, and ability to use nonclinician interviewers suggest that the CONES technique may be an effective complementary strategy for collecting data about perioperative anesthesia events. Perhaps ultimately a modified combination of the two will be shown to improve event detection and analysis, thereby enhancing continual quality improvement processes.

Previous research in this area has examined both intraoperative as well as recovery room occurrences that fell on a continuum of adverse anesthetic-related events. In the published work perhaps most similar to ours, Cooper et al.  17asked providers to self-report events on patient admission to the postanesthesia care unit on a form listing approximately 90 possible events (as opposed to being collected via  facilitated CONES interview). Like CONES, the authors used a wide definition of “recovery room impact events,” and these were collected anonymously and contemporaneously. The incidence of recovery room impact events in their study was 18%, approximately half of what we detected. Similar to our findings, recovery room impact events with more than minor sequelae were infrequent (2.5–3%). In contrast, in an even earlier unpublished 1983 study by R. W. Vaughan, M.D., et al.  at the University of Arizona, a trained investigator observed 451 adult cases and reported that 46.6% of these cases contained at least one “anesthesia-related consequence” (with a definition similar to the “critical incidents” of Cooper et al.  11,12). Therefore, differences in anesthesia event reporting rates include differing definitions of an event as well as methods of identifying those events. An important distinction in our approach from many previous event detection methods is our use of trained examiners actively interviewing clinicians using a progressive cascade of questions and a strict methodology.

The CONES was well received by anesthesia providers at the study hospital. The interview process was not perceived as a threat to autonomy or a hindrance to efficiency. The anonymous process of reporting NRE was appreciated by anesthesia providers to be stress free and had the potential to identify clinical system or process problems. The active surveillance technique using trained interviewers and a structured survey instrument facilitated data collection and hence improved compliance and clinician acceptance. The addition of CONES may help to create a less threatening local event-reporting environment, analogous to the successful model used by the aviation industry to improve flight safety by anonymous submissions to the Aviation Safety Reporting System.§§

The CONES was administered in close proximity to the care provided, and the relevant data were collected actively (through interviewer-facilitated structured surveys). Therefore, CONES data collection can be distinguished from traditional QA self-reporting in that (1) CONES is open-ended (i.e. , any type of event can be reported), whereas QA typically uses prescribed categories; (2) CONES data are collected actively, whereas traditional QA systems rely on voluntary self-reporting and/or chart review; and (3) CONES contemporaneously collects data about event etiology, detection, management, and possible contributory factors, whereas in traditional QA systems, such data are only obtained days or weeks after the event is reported. Although both techniques are retrospective, the contemporaneous nature of CONES review may be less prone to “cognitive contamination or bias” that tends to develop with the passage of time and knowledge of the outcome. In addition, the CONES data may be more accurate because it comes directly from the frontline provider immediately after the case as opposed to later via  chart review. The literature suggests serious shortcomings of delayed data collection, with postevent recall and event documentation being significantly affected by memory shaping, hindsight bias, and outcome bias.9,18,19 

CONES data were statistically indistinguishable from the surgical population as a whole in regard to age, sex, ASA physical status, type of anesthesia used, and type of anesthesia provider. In contrast, the QA cases involved sicker patients (higher ASA physical status), were more often emergencies (twice as common as seen in the population), and were reported more often by anesthesia residents than by certified registered nurse anesthetists. As a result, cardiothoracic, vascular, and neurosurgical service patients were represented disproportionately among QA cases.

Events captured by traditional QA self-reporting systems (whether voluntary or “mandatory”) tend to emphasize sicker patients undergoing high-risk surgeries and thus may not provide a reliable picture of the risks to anesthesia safety. The care of very ill patients is complex, and when adverse events occur, numerous factors can obscure the identification of remediable causal variables. Correcting process deficiencies discovered from analyzing events in healthier patients undergoing elective procedures may be more productive and nonetheless benefit the sicker high-risk patients who have a lower tolerance for complications or system failures.

Primarily because of the way CONES data were collected, emergency cases and three types of surgery (cardiothoracic, ophthalmologic, and plastic surgery) were underrepresented. The CONES surveyors rarely encountered emergent patients because most of these cases were completed after the daytime work shift. Similarly, cardiothoracic cases were usually long procedures ending late in the day and postoperatively were transferred directly to the surgical intensive care unit. Almost all ophthalmology patients also bypassed the postanesthesia care unit, being recovered in the same-day surgery unit. Why plastic surgery cases were overrepresented in the CONES sample is unclear. Some of these sampling issues could be remedied by altering the location and work schedule of the CONES surveyor.

Self-reported QA data has been consistently shown to underestimate the occurrence of patient injury.20–22The CONES method confirms this shortcoming of traditional event reporting. Extrapolating the CONES findings suggested that 7.7% of all patients undergoing anesthesia may be injured, albeit mostly of nominal severity. Half of QA injuries reported were also minimal in severity (1–2 on the Outcome Severity Score). Although the injury severity of QA data was significantly greater than that of CONES data, the small number of CONES cases with injury (n = 13) may be insufficient to assure statistically stable results. Notably, two cases with injury in the CONES database were excluded from this analysis because they were also captured in the QA data. Recalculated injury rates with those cases included suggest an incidence of 8.3% for CONES and 1.2% for self-reported QA data.

The QA process tended to capture the more serious patient injury events, whereas the CONES method captured a larger number and wider variety of events. Although the CONES events tended to be mild to moderate in terms of actual patient injury, they may nonetheless be significant harbingers of patient care quality and satisfaction. Traditional event self-reporting has not typically been a reliable method of capturing near misses. Often, noninjurious events are underreported because it is a common perception that such events do not significantly contribute to patient outcome and are therefore not worthy of reporting. However, near-miss events can provide valuable insights regarding potential sources of system failure and risks of future injury events.10 

The CONES results demonstrate that many patients undergoing anesthesia experience significant physiologic perturbations, most of which do not lead to overt perioperative injury. Almost one quarter of CONES patients experienced such an impact event. Of note, among cases with NREs, QA cases averaged two patient impact events, whereas CONES cases were more commonly singular events. Some might dismiss such impact events as unimportant because they are not “outcomes.” However, they may be harbingers of later postoperative adverse events. Moreover, we suggest that even if transient myocardial ischemia, oxygen desaturation, or profound hypotension, for example, do not cause obvious tissue injury, they represent “at-risk” situations, which are undesirable.23–25The increased frequency of impact events among more severe QA cases gives some credibility to the assertion that these patient impact factors play a role in adverse outcomes.

We examined which putative contributory factors were present in cases with NREs (table 4). Factors such as preexisting patient disease weighed more heavily in QA data, suggesting that self-reported QA data and its peer-review analyses tend to emphasize patient phenomena rather than clinical or systems processes. Equipment and environmental factors, which were much more frequent in CONES cases, are potentially important remediable latent conditions. The relative absence of such factors in the QA data may reflect either a failure to capture information about latent conditions in traditional QA processes or reluctance of anesthesia providers to report them. Training of clinicians to recognize and report mutable contributory factors may be beneficial.

To some extent, CONES, like traditional event self-reporting, is vulnerable to individual practice patterns, behaviors, and cognitive biases. It can be difficult for clinicians who are not trained in event recognition to identify and analyze evolving event situations. Individual clinicians show tremendous variability in reporting styles.21,26Explicit training of clinicians and support staff in NRE recognition and description may be useful. The clinical reviewers in our study were authors of this article and therefore represent a potential analytical bias.

Because both approaches are retrospective in nature, they can be influenced by both outcome bias—“the influence of outcome knowledge on evaluations of decision quality”27—and hindsight bias—“the tendency for people with outcome knowledge to exaggerate the extent to which they would have predicted the event beforehand.”9,27,28QA data, which tend to emphasize more serious adverse events, are particularly prone to these biases. In addition, QA event submission often involves many days of follow-up, analysis, and discussion with peers and superiors, during which time conclusions may be drawn prematurely or shaped by “conventional wisdom” about what happened and why. The assessment of causality and culpability is strongly influenced by knowledge of the outcome.29,30Events with bad outcomes are more commonly judged “serious,” whereas events with minor outcomes are viewed as unimportant. However, these “minor” events represent an opportunity to identify the latent conditions that pose a threat to future patients. The nature of CONES focuses attention on process, thus providing insight on mechanisms of failure rather than the failure itself. Therefore, the effects of outcome and hindsight bias may be minimized.

The reduced effort required for clinicians to report events with CONES, and the fact that data collection occurs immediately after the event, may similarly reduce recall bias (the human tendency to only remember unique or novel events) and reporting bias (the reporting or documentation of such events).31CONES, however, can be affected by postevent cognitive shaping (e.g. , rationalization, justification).18,32For example, the number and type of NREs reported could be decreased if providers rationalized some types of events as “routine.” An NRE for one clinician may be more routine for another. For example, a provider who had not been properly trained in the use of a medical device may frequently have trouble using that device but might be less likely to report events related to this misuse as an NRE.

These potential biases could still be reduced further. If data were collected prospectively (e.g. , through videotaping or direct observation of clinical care11,33,34), there would be an independent mechanism to detect and validate NREs.12Second, the CONES surveyor could use specific probes and questions to elucidate events or practices that deviate from standards of care. Third, clinicians could be given training and reminders to increase their awareness of and willingness to report NREs. In contrast, self-reporting systems are prone to other problems, including providers failing or forgetting to report significant events or details, filling out forms prematurely (e.g. , before the end of the case), or completing forms hastily or incompletely because of production pressure. The face-to-face interactive nature of CONES has the potential to yield a larger amount of high-quality event data.

Our laboratory has videotaped hundreds of anesthesia cases,35as have Mackenzie et al.  36,37in the trauma suite, and others.38Using prospective video analysis of cases at the same institution, the incidence of NREs (approximately 35%) was remarkably similar to that found with the CONES method reported here,34,35thereby providing additional validation of the CONES approach as a reliable method of capturing clinical events. The collection, archiving, processing, and analyses of video records is complicated and resource intensive,35perhaps relegating these methods for the present to research applications. In contrast, CONES may be a practical method of everyday clinical quality improvement.

Traditional hospital QA systems tend to focus on sharp-end issues and do not adequately evaluate the potential impact of latent failure modes, including equipment, environmental, and system factors. The structure of the CONES data collection and analysis methods were designed to more readily capture these types of contributory factors. It would be possible to redesign QA systems to emphasize such factors to a greater degree. This would be a valuable direction and is supported by some early results by others with human factors- or systems-oriented morbidity and mortality conferences, for example.

A disadvantage of the CONES method is that the limited amount of data that can be collected about the nature of the event because of production pressure (i.e. , getting the next case started) precludes exhaustive analysis of individual cases. This could be obviated by follow-up, perhaps using detailed structured interviews,39but these would require substantially more resources. In addition, the CONES method does not capture any patient injuries that manifest after handover of the patient in the postanesthesia care unit. This is a serious limitation of the injury component of the current study and must be addressed in future studies (e.g. , through 30-day patient follow-up).

The purpose of this study was to establish feasibility and potential value. Although future studies will be necessary to evaluate formally the cost-effectiveness of CONES as an adjunct to traditional QA methods, several observations are warranted. The CONES surveyor in this study did not have formal medical training. If sufficient safety information is generated, this additional cost of personnel may prove to be justified. In many hospitals, existing QA personnel could be used to collect and analyze data using the CONES approach. Intermittent or random sampling seems effective and would reduce the overall cost of the technique. Although the cost of collecting the events was nominal, the cost of analysis of the captured events was significant because we had to create the infrastructure and compensate the clinician reviewers.

Unbiased analysis of clinical events by domain experts requires appreciable time. CONES data can theoretically be analyzed more quickly than traditional QA reviews, because CONES data are more circumscribed and can be more readily collated in a standardized structured format. On the other hand, in its current format, CONES data lack the rich detail of chart-review based QA narratives. However, a CONES database is potentially more amenable to quantitative analyses, and the descriptive statistics could be automated.

We describe a method of collecting data about undesirable clinical events that seems to yield new information compared with traditional QA methods. The CONES approach of active surveillance is analogous to strategies already in the public health sector, where passive self-reporting systems are widely appreciated to be unreliable and therefore active surveillance methods are commonly used to augment established processes.40 

CONES introduces a systems approach to anesthesiology quality improvement advocated by patient safety experts.29,41–43A larger volume of potentially useful data are generated, and reporting compliance is improved. The method prompts provider recall, minimizes the burden of interrupted workflow, and creates a nonjudgmental climate within which to report events by respecting provider autonomy, confidentiality, and workload. CONES emphasizes a systems perspective in data collection, analysis, and process improvement.

The typically less severe events captured with the CONES method represent valuable QA information because they (1) are much more frequent and represent a “signal” that can be detected and followed over time in response to QA interventions; (2) may be precursors to or illuminate underlying causes of/contributors to more serious future events; and (3) may have a greater overall impact on patient satisfaction and other “pay-for-performance” anesthesia indicators. In summary, CONES may be an effective complement to existing anesthesiology quality improvement programs.

The authors thank Oralia Loza, M.S. (Biostatistician, University of California, San Diego), for statistical assistance; Samuel Hughes, M.D., Ph.D. (Technician, University of California, San Diego), Amanda Lamond, B.S. (Technician, University of California, San Diego), Amy Groom, B.S. (Technician, University of California, San Diego), and Zobeida Torres, B.S. (Technician, University of California, San Diego), for data collection; Karen Ruberg (Administrative Assistant, University of California, San Diego) and Kelly Vo, B.S. (Administrative Assistant, University of California, San Diego), for technical assistance; Susana Hazelden, B.A. (Editorial Assistant, Clinical Investigation Department, Naval Medical Center San Diego, California), for editorial assistance; and all of the clinicians who participated in the research (San Diego, California).

1. If you were to imagine the perfect case involving this particular patient, were there any deviations from ideal?

2. Did you make any unanticipated or unplanned actions or interventions?

3. Were any unusual requests or demands made on you?

4. Was there anything distinctive about other peoples’ actions or interactions with you?

5. Did anything occur that affected your workload or vigilance?

6. Did anything occur that was annoying or distracting?

7. Did your tools and equipment perform as expected?

8. Did the patient do anything that was out of the ordinary or notable?

9. Did the patient respond as expected to your treatment and interventions?

If any nonroutine events occurred during the case, the following additional questions were asked:

• 10. Please briefly describe the event.

• 11. What were your initial clues that something was atypical?

• 12. Why did it occur?

• 13. How did you respond (if you did)?

• 14. What factors, issues, or other events influenced your response?

• 15. What was the result of your intervention?

• 16. Was your attending present when the event happened? YES NO

• 17. If not, was your attending made aware that the event occurred? YES NO

• 18. Was there any patient injury? YES NO

• 19. Was the injury (injuries) preventable? YES NO

• 20. What could have been done to prevent injury?

• 21. If there was no patient injury, estimate the percent likelihood that the event, if not managed correctly, could have led to patient injury (0–100%).

• 22. Did the nonroutine event affect the patient’s perioperative course in any significant way? That is, did it result in, for example, case cancellation, or a change in postoperative management or patient disposition (intensive care unit instead of ward admission)?