We appreciate the comments of Gurnaney et al. regarding the design and scoring of simulation scenarios. As Gurnaney et al. indicate, the sequence of actions is often important. The scoring checklist in this study did not incorporate a sequential approach for actions to be credited and did not involve the subtraction of points for actions that should not have been performed. Although a variety of scoring methods have been used and their reliability reported, we selected a checklist scoring method in this study and reported the reliability of scores. The primary reason not to incorporate a sequential approach or a weighted scoring was to simplify the scoring mechanics. These are some of the challenges that have been described in scenario design and checklist creation.1
Regarding the potential bias introduced by prior exposure to simulation, the trainees in this program had extensive simulation experience during their training, but at the time of this study it had been limited to adult mannequins and task trainers. Their pediatric specific simulation experience was therefore limited.
As Gurnaney indicates, some of the scenarios, such as malignant hyperthermia and asthma, are common to adult practice. Experience garnered in the adult realm can certainly translate into pediatric practice. We agree this may explain why these scenarios were not effective at differentiating participants with more from those with less pediatric training experience. An important observation from this study is that some participants did not perform at the highest levels despite having greater clinical experience.2Even though some scenarios might not be as discriminating, the recognition and management of these conditions is considered essential to pediatric anesthesia practice and add to the catalog of scenarios that can be utilized to assess an individual's performance in a stressful but standardized situation. The feedback obtained from participants as well as the scenario discriminations are helpful in selecting and designing a set of pediatric scenarios that could be used in a multiple-scenario pediatric anesthesia assessment.
Simulation holds the promise of being a component of multi-modal assessment of the ability of residents and fellows at pediatric anesthesia skills, providing reproducible scenarios that do not put our most vulnerable patients at risk.
