Key words: Complications. Monitoring. Quality assurance.
In this issue of Anesthesiology, another study adds to evidence that quality assurance (QA) programs based on voluntary reporting identify only a small fraction of critical events. [1]We learn that anesthesiologists, as has been shown for other clinicians, do not report most events meeting criteria that they themselves had defined as relevant to QA. This shouldn't be surprising. Why should we expect clinicians to report their own adverse outcomes if reporting might jeopardize their career? The prospect of public embarrassment or retribution could be a sufficient deterrent to satisfactory self-reporting. Perhaps equally dissuading is that the benefit to patient care of anesthesia QA systems has not been established rigorously, especially for QA indicators such as deviations in vital signs. This study also indicates the potential for using the Automated Anesthesia Record for Quality Assurance: Is this tool sufficiently practical and useful to be worth the cost for this application?
Sanborn et al. [1]provide data that question the effectiveness of voluntary reporting to measure accurately the true incidence of undesirable anesthesia clinical events. Using a commercial Automated Anesthesia Record (AAR, essentially equivalent to an Automated Information Management System), they documented deviations from predefined limits of vital signs. These deviations had been agreed on by their department for its QA program. Eliminating artifacts and nonclinically-significant deviations, they found 434 intraoperative events within 5,454 anesthesia records. Relatively few of these events (4.1%) were reported in a parallel, electronic system that clinicians used for entry of QA data. They further demonstrated that these vital signs' deviations were associated with (although not necessarily the cause of) intraoperative and postoperative mortality.
Several other studies support the argument that QA programs based on voluntary reporting detect relatively few of all events. Most recently, Cullen et al. [2]reported that only 6% of hospital adverse drug events were reported through the hospital incident reporting system, and in other studies, researchers reported similar findings. [3,4]There is some evidence to the contrary (e.g., an innovative, confidential program for voluntary reporting by house officers appears to have been efficacious. [5]As many events were reported as were identified via a chart review and at a lower cost). But, there are good reasons to believe that voluntary reporting does not reliably capture the true nature or extent of adverse events. Sanborn et al. offer some possible reasons for the differences between the electronic record and voluntary reporting. Yet, without a measure from the participants (e.g., via a survey or qualitative study), we are left to speculate why the clinicians didn't report more of the events. A general explanation is that better incentives are needed to encourage anesthesiologists to report their outcomes, mistakes, and other system problems. At least two steps toward that are necessary: 1) demonstrate the value of reporting, and 2) change the culture that attributes error to negligence.*
The Value of Reporting Events
Tracking substantive deviations in vital signs should, in principle, be useful to indicate something about the process of anesthesia care. But, the value in documenting these events, whether reported voluntarily or by an AAR, has yet to be demonstrated. Will decreasing the rate of hypotension, hypertension, tachycardia and bradycardia, hypoxemia and hypothermia, presuming that it can be done, make any difference to patient outcome? There is some anecdotal evidence that comprehensive use of an AAR is associated with reductions in occurrence of such indicators and even of indicators that are not documented automatically.** But, there still is no well controlled study to suggest that this will result in reductions in the rate of serious adverse outcomes. Perhaps it is better care if rates of events such as hypotension or hypoxemia of lasting duration are reduced from some baseline; common sense suggests that flying further above the trees provides a larger margin for error. Or, perhaps these events are just normal perturbations. If nothing else, reporting the rates among centers that use AAR systems will give comparative data for benchmarking between hospitals and clinicians. Sanborn et al. and others with access to an AAR have the opportunity to lead in this effort. They must examine not only event frequencies, but also measure what it takes to improve performance indicators, with or without the assistance of expensive technology. There actually is some evidence that self-reporting can provide useful information for anesthesia quality improvement if the criteria for reporting are set by the individual clinician for the individual case. [6]To be effective, such a program requires investigation of system issues and a greater expenditure of resources to learn which events are important and to devise remedial strategies.
Although it was not one of the objectives of the study, it may have been helpful had the investigators examined the most serious outcomes more closely, to identify patterns or system problems. The information in the AAR may have provided new insights. There probably is a payoff to examining such sentinel events [2]-the Australians think so. Their legal system protects from disclosure; they collect data and analyze it via study committees. They have shown improvements in outcome during the 30 years this has been done. [7]As Gaba [8]advocated for anesthesia and Leape [9]more recently for medicine and health care in general, it is the identification and correction of system problems that is more likely to yield benefits. But, for that to happen, a culture change must be made.
The Culture of Reporting Events
The culture of health care is not forgiving of errors by clinicians, especially physicians. As Leape wrote recently,
". . .In everyday hospital practice, the message is .. clear: mistakes are unacceptable. Physicians are expected to function without error, an expectation that physicians translate into the need to be infallible. . . .This kind of thinking lies behind a common reaction by physicians: How can there be an error without negligence?" [9]
Neither the public nor physicians themselves are tolerant of medical error; indeed, physicians generally do not feel comfortable talking about their errors, nor can colleagues generally be expected to offer support. [10,11]
For meaningful change toward more openness in reporting errors and system problems, the culture must change. Deviations and errors need to be accepted as the inevitable result of imperfect systems and the normal limitations of men and women. A culture of safety understands that reports of problems, errors, and accidents are the material from which learning and improvement are born. Fortunately, there is a growing body of literature in which this important topic is being aired.*** [9,11]But, more open dialogue about error is needed for quality assurance to have its intended impact.
Without such a substantial change in the culture of health care, voluntary reporting of events will continue to underperform. Perhaps what is needed is more protection from unwarranted public disclosure, as is afforded to pilots through the Aviation Safety Reporting System or for anesthesiologists in Australia who report through the Australian Incident Monitoring System. [12]Can clinicians in the United States be protected from retribution for reporting problems for which they may have some responsibility? Despite efforts to offer such protection, there has not yet been effective action to motivate increased levels of reporting and discussion of adverse events. In New Jersey, there is a ray of hope. A state law that requires reporting of adverse events also calls for protecting the events from public disclosure. Although there has not yet been a court challenge to test the law, a recent request for disclosure was rebuffed.**** We are probably a long way from a national system to report medical error and system problems similar to the Aviation Safety Reporting System. In the meantime, development of local reporting systems, with appropriate legal protection, should be an attainable and useful goal. Those systems must include provisions for analysis and feedback of the information and for instituting system changes that generate real improvements in the system for patient care.
The many potential benefits of the AAR have been described. [13,14]An AAR system was a necessary instrument for the current study, and its use uncovered deviations associated with mortality. Should departments now rush to purchase this costly technology? Manual recordkeeping itself is suspect for documenting events. Substantial discrepancies have been demonstrated between automatic and manual reporting of intraoperative blood pressure. [15]Yet, the greater accuracy, neatness, and ability to track noncritical events are unlikely to be sufficient motivation to justify the current cost of approximately $6 per procedure.***** Because a benefit for outcome or quality cannot be shown clearly, implementing an AAR will continue to have a tough battle when competing in the new corporate environment of health care. Quality assurance alone will not be the driving force for the AAR except for the more forward-thinking administrators. Its acceptance is more likely to be market-driven. As a hospital recognizes that it needs information about utilization of the operating room resources, as many already have, automated data-collection will be the only way to get that reliably and consistently. Anesthesia departments will be more driven to modernize their production facility when the hospital information system really works and can connect to it. Many are already doing so; it is likely that all will in the not-too-distant future. Quality assurance may well be a beneficiary.
Can the AAR be useful and cost effective as a tool for QA? Sanborn et al. provide evidence of the potential for that. What needs to be shown now is how the data can be used to direct and stimulate effective and long-lasting changes. But, that will not be so easy. In the larger arena of outcomes research, despite efforts to do so, it has proven difficult to design effective tools to consistently change behavior. As has been said about motivating behavior change for compliance of national guidelines for care, ". . .it is unlikely that progress will be made . . . ..without input from the fields of behavioral science and organizational behavior." [16]
When it comes to getting human error out in the open and making patient safety a priority, anesthesiology is the leading medical specialty. Despite the deficiencies in reporting systems, anesthesiologists can feel pride in all they have done to assume responsibility for iatrogenic outcomes. The development of the AAR and the publication of studies such as Sanborn et al. in Anesthesiology are further demonstrations of a historical commitment of this specialty to identify and talk about its contribution to adverse outcomes. More of the same is needed to lower barriers that continue to dissuade clinicians from openly revealing the imperfections that are the data for learning how to improve.
Is it time to abandon self-reporting of incidents or events as a QA tool? Probably not. Even if only a few percent of actual deviations are reported, some important problems are likely to surface. [2]These reports will come from those who recognize the importance of fixing system problems and who are willing to identify incidents in which they have been involved or even helped to cause. Some departments seem to have successfully implemented QA programs using voluntary reporting. [6]But, as Sanborn et al. suggest, the data probably do not represent a thorough picture of the extent of the problem. In addition, effective monitoring and improvement in quality must be based on a process that includes thorough investigation of sentinel events, feedback of data to individuals, and analysis of system problems. Most of all, a change is needed in the culture of all of health care. Rather than assigning blame, we must encourage open discussion of problems and protect clinicians from unwarranted public scrutiny. Reward people for doing the right thing: being honest about mistakes and so working to prevent them from happening again.
Jeffrey B. Cooper, Ph.D., Department of Anesthesia and Critical Care, Massachusetts General Hospital, 55 Fruit St., Boston, Massachusetts 02114.
* Heylan R: The relevant cause in anesthetic death. Critique on concrete singular causal opinions in research on avoidable anesthetic mortality and morbidity highlighted by legal and philosophical causation theories. R. Heylan, St. Jansziekenhuis, Schiepse Bos 2, B-3600 Genk Belgium, 1995 (PhD dissertation).
** Edsall DW, Deshane PD: Getting the most out of an information system: Annual meeting of the New England Society of Anesthesiologists, Chatham, Massachussetts, September 27-28, 1996.
*** Heylan, op. cit.
**** Barbara Andrews, NJ State Department of Health, and Ervin Moss, M.D.: (personal communicaton).
***** Basis of estimate: $20,000/rm; 7 yr useful life; 5% annual service and software maintenance; 10% annual administrative management cost; total of approximately $6,000/yr/rm; for 1,000 procedures/rm, [nearly equal] $6/procedure.
