UNDOUBTEDLY, many readers will find it curious to see an article1summarizing perceptions of teamwork in the operating room (OR) in Anesthesiology, a journal typically filled with anesthesia-related clinical and biomedical science. Although we would hopefully all agree that teamwork is important in the increasingly complex OR environment, many will wonder how subjective perceptions of teamwork can be turned into a metric useful in improving care. And, most important, how can such seemingly vague assessments of teamwork ultimately benefit our patients?
Concern about teamwork in complex healthcare settings is not new. The now-familiar 1999 Institute of Medicine report,2which highlighted anesthesiology’s progress and leadership in patient safety, recommended that healthcare organizations enhance their patient safety culture, in which “effective team functioning” was identified as one of five principles for creating safe hospital systems. In response, at least nine surveys purporting to measure patient safety climate have emerged, of which the Safety Attitudes Questionnaire (SAQ) seems most robust.3A modification of a survey developed for critical care settings from an aviation survey, SAQ is a 30-question survey designed for intrainstitutional and interinstitutional comparisons. Its six domains—teamwork climate, safety climate, job satisfaction, perceptions of management, stress recognition, and working conditions—reflect distinct dimensions based on studies in aviation and health care. Higher SAQ scores have been associated with shorter duration of stay, fewer medication errors, and lower bloodstream infection and ventilator-associated pneumonia rates, as well as lower nurse turnover and risk-adjusted patient mortality rates.
The article1in this issue of Anesthesiology evaluates how well the teamwork climate domain in an SAQ version enhanced for OR personnel reflects the personnel’s perceptions of teamwork climate, which is defined here as the quality of collaboration among those in the work setting. Measuring teamwork climate is challenging principally because it is a concept or construct that cannot be measured directly; instead, it is assessed indirectly by asking, in this case, 6 of the 30 questions (or “items”) in the SAQ survey (or “instrument”). The responses to the 6 questions are converted to numerical scores that, in turn, are combined and arrayed on a 0–100 scale as a “domain score.” Although the methodology presented may seem arcane, it relies on the same item-measurement theory on which more familiar generic and disorder-specific quality-of-life scales (e.g. , Medical Outcomes Study Short Form-36 Health Survey, visual analog scale for pain) are based.4,5If the items have been selected appropriately and the scaling has been done properly, the resulting domain scores should reflect the underlying construct, should differ in a given circumstance by only random error of measurement, and should have important properties that provide the basis for the evaluation.4,5
Among those properties is coverage of the construct: All aspects of teamwork climate that are meaningful to OR personnel should be addressed. This seems to have been satisfied in the development of the modified SAQ through the use of literature review, focus groups, behavioral observation of OR personnel, and critiques of the draft survey by OR personnel. A second property is reliability , which in its most general usage is the consistency of a measurement under constant conditions. For this instrument, two specific types of reliability were tested: Internal reliability, or the consistency with which raters used the 6 items in making their assessments of teamwork climate, was evaluated using Cronbach’s α, whose values were uniformly high for the different caregiver types. Consistency with which personnel of each type made their assessments was determined with an intraclass correlation coefficient, whose values were similarly high. The third property is validity , a multidimensional characteristic that, overall, means the domain measures what it is claimed to measure. Given the absence of an existing teamwork metric for use as a comparator, it is not possible to explore correlations with other ways to measure teamwork; therefore, we cannot assess convergent, criterion, or discriminant validity.
Potential users of this instrument will be especially interested in its ability to satisfy a fourth property, sensitivity : the instrument’s ability to reflect true differences in teamwork climate. The investigators used the instrument in 60 hospitals in a large health system, achieved high participation rates among all OR personnel types, and found widely varying perceptions of “good teamwork climate” by hospital. All respondents in 6 hospitals agreed that a good teamwork climate was present, whereas in the majority of study hospitals, less than 50% of respondents shared that assessment. Given aforementioned psychometric characteristics, this phenomenon suggests that the instrument is sensitive to differences in teamwork climate. The instrument was also sensitive to perceptions among different caregiver types after adjusting for hospital, although differences were much smaller. Using the same data, the investigators probed deeper into differences in caregiver-specific assessments in a recent publication,6yielding fascinating perspectives on OR teamwork climate: Anesthesiologists believe that they enjoy good collaboration with other anesthesiologists near uniformly and with both nurse anesthetists and OR nurses almost as commonly, whereas with surgeons, good collaboration is much less prevalent. A high proportion of surgeons think they have good collaborations with all other OR caregivers. In contrast, both types of nurses rate teamwork with other caregivers much more poorly than the physicians do, with only approximately half of nurses believing that they enjoy good collaboration with surgeons and a slightly higher proportion regarding the teamwork climate with anesthesiologists favorably (table 1).
Related to sensitivity is the fifth property, responsiveness : the instrument’s ability to reflect true change in teamwork climate. Clearly, if this instrument is to be useful as a metric in improvement work, it must be able to document changing perceptions as the teamwork climate improves. That this instrument can reveal differences in teamwork climate across 60 hospitals in a cross-sectional study design is evidence of sensitivity but not necessarily responsiveness. A longitudinal design, with repeated measures using this instrument as an improvement initiative progresses, is needed to demonstrate responsiveness.
Even if this instrument is shown to be responsive to change in teamwork climate, we must not forget that the “culture of safety” described in the Institute of Medicine report2is very much multidimensional. As we continue to model our efforts on the successes that occurred in aviation,7we should not forget that the aviation industry’s culture of safety not only encourages good communication but also includes a fairly rigorous regulatory approach that is not present in medicine. Although both pilots and medical residents cannot work beyond a specified number of hours, only pilots must meet minimum sleep requirements to be able to work, are trained in fatigue management, and more likely to be selected for random drug screens. Although both pilots and most physicians must undergo recertification to demonstrate proficiency in their craft, pilots do so much more frequently, using more rigorous simulation techniques to test knowledge, and face mandatory retirement. Although both pilots and physicians have strong concerns about the safety of their charges, only pilots have mandatory safety reporting systems in place, use national systems to evaluate and learn from adverse events, and suffer the same consequences as their charges when adverse events occur.
Improving communications and satisfaction is an important step in achieving a safety climate8in the OR. However, achieving a culture of safety in health care is likely to require additional steps to have a documentable and lasting impact on patient outcomes.
* Yale University, New Haven, Connecticut. Clinical Epidemiology Research Center, West Haven VA Connecticut Health System, West Haven, Connecticut. fred.orkin68@post.harvard.edu.† VA National Quality Scholars Fellowship Program, Veterans Affairs Medical Center, White River Junction, Vermont. Health Services Research & Development Enhancement Award Program, and White River Junction Field Office, VA National Center for Patient Safety, Veterans Affairs Medical Center, White River Junction, Vermont. Departments of Psychiatry and of Community and Family Medicine, Dartmouth Medical School, Hanover, New Hampshire.
