Background

Previous studies have indicated that fiberoptic orotracheal intubation (FOI) skills can be learned outside the operating room. The purpose of this study was to determine which of two educational interventions allows learners to gain greater capacity for performing the procedure.

Methods

Respiratory therapists were randomly assigned to a low-fidelity or high-fidelity training model group. The low-fidelity group was guided by experts, on a nonanatomic model designed to refine fiberoptic manipulation skills. The high-fidelity group practiced their skills on a computerized virtual reality bronchoscopy simulator. After training, subjects performed two consecutive FOIs on healthy, anesthetized patients with predicted "easy" intubations. Each subject's FOI was evaluated by blinded examiners, using a validated global rating scale and checklist. Success and time were also measured.

Results

Data were analyzed using a two-way mixed design analysis of variance. There was no significant difference between the low-fidelity (n = 14) and high-fidelity (n = 14) model groups when compared with the global rating scale, checklist, time, and success at achieving tracheal intubation (all P = not significant). Second attempts in both groups were significantly better than first attempts (P < 0.001), and there was no interaction between "fidelity of training model" and "first versus second attempt" scores.

Conclusions

There was no added benefit from training on a costly virtual reality model with respect to transfer of FOI skills to intraoperative patient care. Second attempts in both groups were significantly better than first attempts. Low-fidelity models for FOI training outside the operating room are an alternative for programs with budgetary constraints.

THE emergence of simulation-based training techniques provides a new form of teaching intervention that may allow anesthesiology trainees the opportunity to learn complex psychomotor tasks more efficiently.1Fiberoptic orotracheal intubation (FOI) is a complex psychomotor task essential to the practice of anesthesiology. Therefore, simulation training for FOI may be of benefit to anesthesiology training programs.2But, as with any new technology or technique; validation is essential to determine the safest and most efficient teaching method.3

Despite being regarded as an essential skill in anesthesiology, there are no formal guidelines for the training of anesthesiology residents in FOI.4,5There is also a paucity of educational research data validating any particular educational intervention as an effective means of teaching FOI. Previous studies have indicated that FOI skills can be learned outside the operating room, and that skills acquired using FOI on healthy, anesthetized, apneic patients transfer to other, more urgent clinical situations.5,6The purpose of our study was to determine which of two relatively new educational interventions for teaching FOI allows learners to gain greater capacity for performing the procedure. Two simulation-based teaching interventions for teaching FOI were compared. The first intervention involved training on a low-fidelity model that allowed trainees to practice the manual movements and hand–eye coordination required to perform FOI using only a standard fiberoptic bronchoscope and a specially designed wooden box.6

The second teaching intervention under consideration is the AccuTouch Flexible Bronchoscopy Simulator (Immersion Medical, Gaithersburg, MD). This virtual reality device provides a high-fidelity simulation of both the manual movements and the hand–eye coordination required to perform FOI. It is a fairly accurate simulation of human airway anatomy in that its components include a mock video-laryngoscope head that provides haptic feedback and is virtually indistinguishable from an actual video-laryngoscope head. The model also has a video monitor that displays a realistic simulation of airway anatomy, including realistic renderings of human oropharyngeal tissue and anatomical structures, including the oral mucosa, tongue, epiglottis, tonsillar pillars, esophagus, and vocal cords. This high-fidelity model has previously established construct validity as a bronchoscopy teaching tool.7,8

This study does not include a group trained using only traditional didactic teaching methods, because another study by our research group used very similar test conditions and measurement tools to establish that traditional didactic training for FOI shows little training benefit in the absence of real-world experience or some degree of simulation training.6This study represents a first report comparing high-fidelity versus  low-fidelity simulation training for a complex psychomotor task with training results tested in live human patients in a clinical setting.

This study used a prospective, randomized, single blinded, two-arm design. Ethics committee approval was obtained both institutionally from St. Michael's Hospital (Toronto, Ontario, Canada) and from the University of Toronto (Toronto, Ontario, Canada). Written consent was obtained from all subjects and patients involved in the study before their participation. Subjects consisted of registered respiratory therapists (RRTs) from our institution. RRTs are a useful surrogate for junior anesthesia residents in that they have a basic knowledge of airway management and are familiar with airway management equipment but are complete novices in manipulating the fiberoptic bronchoscope. Subjects were excluded if they had independently performed FOI. Anesthesia residents were not used as subjects because of their variable experience with FOI, which would make true randomization difficult and could bias our results.

Thirty RRTs were recruited on a voluntary basis and randomly assigned by a computerized random number generator and sealed envelope technique to one of two groups. One group was trained in the use of a fiberoptic bronchoscope by way of a low-fidelity bench model, whereas the other group was trained in the use of a fiberoptic bronchoscope using an anatomically accurate, high-fidelity computerized airway simulation.7,9During the training phase, all subjects received a short handout providing basic information regarding fiberoptic bronchoscopy. Training of both groups was supervised by one of the researchers, who offered minimal individualized supplemental instruction so as not to bias the study in favor of subjects who had received either modality of training. Subjects were given access to their training modality for up to 1 h, because a previous study and pilot studies have suggested that most subjects require less than an hour to maximize the training effect with either model.6Their training was done individually using a single session with the assigned model and instructor. Because of the number of individual sessions required and scheduling logistics, the training phase lasted approximately 3 months.

Low-fidelity training was performed using a “choose-the-hole” model designed to refine FOI skills in conjunction with the bronchoscope described below.6High-fidelity training was performed using an AccuTouch Flexible Bronchoscopy Simulator.9All bronchoscopes and virtual bronchoscopes used during the training and testing phase of the trial were of the “video bronchoscope” variety, which do not use a viewing eyepiece but rather a computer monitor.

The testing phase of the study involved subjects performing FOI on a real patient after having been trained using one of the aforementioned modalities. Testing was performed within 1 week of each subject's training date. Therefore, the testing phase of the trial was run concurrently with the training phase to minimize the effects of time on the subjects' newly acquired skills.

After ethics approval and informed consent, patients were recruited in the preoperative holding area of our institution. All study patients were under the care of an anesthesiologist not involved in the study throughout the perioperative period. Eligible patients were healthy patients (American Society of Anesthesiologists physical status I or II), scheduled for elective cases requiring a general anesthetic with tracheal intubation, with normal airway anatomy, and with a body mass index of less than 25 kg/m2. General anesthesia was induced by the attending anesthesiologist. The choice of induction agents and muscle relaxant was at the discretion of the anesthesiologist in charge of the patient, but always with preoxygenation and bag–mask ventilation before intubation. In attempting to perform FOI, subjects were assisted with a jaw-thrust maneuver only, and were allowed to proceed for up to 180 s or until the patient's oxygen saturation levels began to decrease below 95%. Testing was performed on live patients with a 5-mm Pentax fiberoptic bronchoscope (model FB-15BS; Mississauga, Ontario, Canada).

Fully trained subjects were given the opportunity to perform FOI on two separate patients on the same day, within a week of their training date. They were evaluated in real-time by two expert evaluators blinded to their training modality using a previously validated global rating scale (GRS), and performance checklist as measures (see appendices 1 and 2).6,10 Secondary outcomes included success rate and time to perform tracheal intubation. The subjects received no feedback from the evaluators between attempts.

### Statistical Analysis

All statistical analyses were performed using SigmaStat (Systat Software, Inc., San Jose, CA) and SPSS (SPSS Inc., Chicago, IL) software packages. Two expert evaluators evaluated subjects during the testing phase. Sample size was calculated using an effect size of 1.2 SDs based on a similar study by the authors.6When one considers the enormous cost differential between these two teaching models, a large teaching effect size should be required to show a significant difference. That is, the expensive device should be much more effective to justify the 500-fold price difference. With 12 subjects in each group, using a β of 0.20 and a two-tailed α of 0.05, we had 80% power to detect an effect size of 1.2 SDs.

Our primary outcome measures (GRS scores and performance checklist scores) were analyzed using a two-way, mixed-design analysis of variance (ANOVA). The independent variables were “fidelity of training modality” as the between-subjects variable and “postintervention attempt number” as the within-subjects measure. A mixed ANOVA was used because our variables were a mixture of “between subjects” and “within subjects.” The mixed ANOVA not only allowed a comparison of the effect of training modality and the effect of number of attempts as main effects, but also enabled us to examine the interaction between these two variables. For example, any effects of training fidelity may be lost if one were to compare the average evaluation scores of both attempts on real patients, but post hoc  analysis allowed us to identify any differences that may have existed between training modalities after only the first attempt on a real patient but were extinguished by the second attempt. Pearson product–moment tests were used to establish the level of interobserver correlation between the two expert evaluators for both the GRS and performance checklists. A two-tailed P  value less than 0.05 was considered statistically significant. Categoric data including success–failure and pass–fail results were analyzed by Fisher exact test. Time to task completion (in seconds) was analyzed using Mann–Whitney U tests.

Patient demographics were not different between the two groups (table 1). Subject demographics, particularly male:female ratio, reflect the institutional demographics of RRTs employed at our hospital (table 2). A total of 28 subjects completed the study. An equal number of subjects were randomly assigned to each training modality: low-fidelity (n = 14) and high-fidelity (n = 14). Subjects had a median experience level of 54 months (interquartile range, 13–96 months) in the high-fidelity group and 90 months (interquartile range, 20–156 months) in the low-fidelity group. There was no statistically significant difference between experience levels in the two groups (P = 0.420).

Table 1. Patient Demographics (n = 56)

Table 2. Subject Demographics (n = 28)

Two subjects were excluded from the study as a result of scheduling problems whereby they underwent the training phase of the trial but were unable to complete the testing phase because of work schedule conflicts. No subjects had ever performed FOI independently, although most of them had extensive experience assisting with FOIs. Both groups were given a maximum of 1 h to train with their assigned model (time to task).

Interrater reliability was strong for all GRS assessments (r = 0.84, P < 0.05) and checklist evaluations (r = 0.90, P < 0.05). In the high-fidelity group, subjects scored a mean of 24.1 points (SD = 7.8) on the GRS and a median of 9.0 points (interquartile range, 7.5–11.0) on the checklist. While in the low-fidelity group, subjects scored a mean of 25.9 points (SD = 9.5) on the GRS and a median of 10 points (interquartile range, 4.0–11.0) on the checklist.

A two-way, mixed ANOVA detected no significant difference between the global rating scores of subjects trained on either the low-fidelity or the high-fidelity model (F1,26= 0.375, P = 0.546). Similarly, there were no differences detected between the checklist scores of subjects trained with the high-fidelity model when compared with the low-fidelity model (F1,26= 2.099, P = 0.159). The FOI success rate was 50% for both training groups. The median time to FOI completion was not statistically significant between groups, with 74.5 seconds (interquartile range, 40.50–180.0 seconds) for the high-fidelity group and 68.5 seconds (interquartile range, 34.0–180.0 seconds) for the low-fidelity group. These results are summarized in figure 1.

Fig. 1. High-fidelity training  versus low-fidelity training, mean ± SEM. Intraoperative global ratings scale, checklist assessment, success rate, and time to completion of fiberoptic orotracheal intubation performance (all  P = not significant).

Fig. 1. High-fidelity training  versus low-fidelity training, mean ± SEM. Intraoperative global ratings scale, checklist assessment, success rate, and time to completion of fiberoptic orotracheal intubation performance (all  P = not significant).

Close modal

A two-way, mixed ANOVA detected a significant difference between first and second attempts (F1,26= 16.830, P < 0.001 on the GRS and F1,26= 8.966, P < 0.01 on the checklist evaluation). The effect of training modality showed no interaction with respect to attempt number (F1,26= 1.348, P = 0.256 on the GRS and F1,26= 2.099, P = 0.159 on the checklist evaluation). Therefore, subjects improved between first and second attempts equally, irrespective of the training modality to which they were exposed. These results are summarized in figure2.

Fig. 2. First attempt  versus second attempt in all comers, mean ± SEM. Intraoperative global ratings scale and checklist assessment of fiberoptic orotracheal intubation performance (*  P < 0.001, **P < 0.01).

Fig. 2. First attempt  versus second attempt in all comers, mean ± SEM. Intraoperative global ratings scale and checklist assessment of fiberoptic orotracheal intubation performance (*  P < 0.001, **P < 0.01).

Close modal

The primary objective of this study was to test whether there is a difference in skills acquisition when training-naive subjects are trained for FOI using a low-fidelity model versus  a high-fidelity model. Their final skills evaluation was conducted in a live clinical environment using a GRS and a checklist evaluation.

We hypothesized that the high-fidelity model, which allows for a far more accurate representation of human anatomy, would bring higher face validity, as well as greater skills transfer from the part-task trainer to the live clinical setting. Previously validated psychological theory suggests that real-world practice is one of the most beneficial training modalities.11High-fidelity simulation attempts to mimic real-world practice. Therefore, one might expect training effects to increase as the level of training model fidelity increases. This was not the case in this study because the high-fidelity simulation did not impart FOI skills any better than the low-fidelity simulator.

Clinical educators have sought the most efficient and economical methods of teaching advanced technical skills to residents outside of the operating room to maximize performance in the clinical setting. Extraoperative courses have been developed by many specialties to improve their trainees' exposure to technical skills before practicing on real patients. These training courses tend to rely on bench models of varying fidelity to simulate the clinical experience.

Researchers in surgical education have examined the training benefits of bench models with evaluation tools such as the Objective Structured Assessment of Technical Skills.10,12–14These researchers determined that the use of a combined GRS and checklist evaluation was both a valid and a reliable tool for evaluating a trainee's skill at performing a given procedure. Using this evaluation model, previous studies have demonstrated that bench models were equally efficacious as animal models, historically a gold standard for high-fidelity training, in terms of training benefits.12Others have used this evaluation method to show that skills acquired on low-fidelity bench models could be transferred to cadaveric models.15Matsumoto et al.  16used GRS and checklist evaluations to compare high-fidelity and low-fidelity training for endourologic skills, but did their final trainee testing on a bench model. Finally, Naik et al.  6took this work into the clinical setting by using a combined GRS and checklist evaluation to demonstrate that skills learned on a simple bench model could be transferred to real patients in a clinical setting. When combined with previously validated psychological theory and the aforementioned studies, one might assume that training on high-fidelity models would impart a greater benefit than training on lower-fidelity bench models when subjects are evaluated in a clinical setting.11Our results do not support this assumption.

The lack of training effect demonstrated by this study may be attributable to several factors. The most likely explanation seems to be that the high-fidelity simulator is still not yet advanced enough to mimic human anatomy and physiology. Therefore, practice with this form of model would not be as effective as practice in a real environment. However, the model still allows for a similar amount of skills transfer as the low-fidelity model. That is, there may be a theoretical level of “fidelity” required before practice with a model is as effective as practice with the real thing—and possibly this model did not meet that theoretical level of fidelity. The question of fidelity could also potentially represent a limitation of our study in that “fidelity level” is an essentially qualitative characteristic, meaning that the models may have lacked enough difference in fidelity for a difference in training effect to be revealed.

Another potentially limiting factor is the difference in experience level between training groups; this difference was approximately 3 yr. However, the groups were assigned randomly, and there was no statistically significant difference between experience levels.

The FOI success rate for both groups was only 50%. This finding is markedly different from a previous study that found a success rate of greater than 90% using an identical low-fidelity training model and rating scales.6The most likely explanation for this difference seems to lie in potentially different amounts of instruction in the training phase of the trial. In an attempt to determine purely the training effects of the models in question, the current study provided minimal didactic training and focused entirely on subjects' self-training with their assigned models, thus minimizing the effects of didactic teaching.

This study was powered to detect differences between the low-fidelity and high-fidelity training groups. However, because subjects were allowed two attempts at FOI on live patients, it seemed appropriate to study differences between first and second attempts for all comers, as well as the potential effects of training modality on improvements between first and second attempts. These were our secondary outcome measures.

The testing phase of this study allowed subjects two attempts at FOIs within less than 8 h of each other (i.e. , within the RRTs' work shift). We theorized that subjects were given enough time between trials to consider mistakes made during their initial attempts, reflect on potential solutions, and integrate those solutions into their second attempts. In this case, reflection represents a form of mental practice whereby a subject can gain insight into personal learning acquired during his or her first attempt and training phase and integrate that knowledge into problem solving during subsequent attempts.17

This study was powered to detect only a large effect size, and therefore, small training effects were not found to be significant. When one considers the enormous cost differential between these two teaching models ($20 US low-fidelity vs.$100,000 US high-fidelity), requiring only a large teaching effect size to show a significant difference seems warranted.

The results of our study suggest that there is little or no added educational benefit from spending \$100,000 for a high-fidelity bronchoscopy trainer. Low-fidelity models for FOI training may be a viable alternative for all anesthesia training programs, particularly those with budgetary constraints.

1.
Gaba DM: The future vision of simulation in health care. Qual Saf Health Care 2004; 13 (suppl 1):i2–10
2.
Helmreich RL, Davies JM: Anaesthetic simulation and lessons to be learned from aviation. Can J Anaesth 1997; 44:907–12
3.
Byrne AJ, Greaves JD: Assessment instruments used during anaesthetic simulation: Review of published studies. Br J Anaesth 2001; 86:445–50
4.
Benumof JL: Management of the difficult adult airway: With special emphasis on awake tracheal intubation. Anesthesiology 1991; 75:1087–110
5.
Cole AF, Mallon JS, Rolbin SH, Ananthanarayan C: Fiberoptic intubation using anesthetized, paralyzed, apneic patients: Results of a resident training program. Anesthesiology 1996; 84:1101–6
6.
Naik VN, Matsumoto ED, Houston PL, Hamstra SJ, Yeung RY, Mallon JS, Martire TM: Fiberoptic orotracheal intubation on anesthetized patients: Do manipulation skills learned on a simple model transfer into the operating room? Anesthesiology 2001; 95:343–8
7.
Rowe R, Cohen RA: An evaluation of a virtual reality airway simulator. Anesth Analg 2002; 95:62–6
8.
Ost D, De Rosiers A, Britt EJ, Fein AM, Lesser ML, Mehta AC: Assessment of a bronchoscopy simulator. Am J Respir Crit Care Med 2001; 164:2248–55
9.
Colt HG, Crawford SW, Galbraith O III: Virtual reality bronchoscopy simulation: A revolution in procedural training. Am Coll Chest Phys 2001; 120:1333–9
10.
Regehr G, MacRae H, Reznick RK, Szalay D: Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med 1998; 73:993–7
11.
Regehr G, Norman GR: Issues in cognitive psychology: Implications for professional education. Acad Med 1996; 71:988–1001
12.
Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M: Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997; 84:273–8
13.
Reznick R, Regehr G, MacRae H, Martin J, McCulloch W: Testing technical skill via  an innovative “bench station” examination. Am J Surg 1997; 173:226–30
14.
Grober ED, Hamstra SJ, Wanzel KR, Reznick RK, Matsumoto ED, Sidhu RS, Jarvi KA: The educational impact of bench model fidelity on the acquisition of technical skill: The use of clinically relevant outcome measures. Ann Surg 2004; 240:374–81
15.
Anastakis DJ, Regehr G, Reznick RK, Cusimano M, Murnaghan J, Brown M, Hutchison C: Assessment of technical skills transfer from the bench training model to the human model. Am J Surg 1999; 177:167–70
16.
Matsumoto ED, Hamstra SJ, Radomski SB, Cusimano MD: The effect of bench model fidelity on endourological skills: A randomized controlled study. J Urol 2002; 167:1243–7
17.
Schmidt RA, Bjork RA: New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychol Sci 1992; 3:207–17

Figure. No caption available.

Figure. No caption available.

Close modal

Figure. No caption available.

Figure. No caption available.

Close modal