Efficacy of an Online Education Program for Ultrasound Diagnosis of Pneumothorax

• Ultrasound diagnosis or exclusion of pneumothorax is sensitive, specific, and highly useful in the perioperative setting

• Educating anesthesiologists to accurately exclude pneumothorax using ultrasound has not been previously examined

• In a study of 79 anesthesia residents and faculty, a high degree of accuracy to exclude pneumothorax was obtained after simple training using a 5-min online training video, and this knowledge was retained for at least 6 months

PNEUMOTHORAX can be diagnosed by physical examination, chest radiography, ultrasound or computerized tomography scanning. Clinical signs and chest x-ray can often be difficult to interpret, whereas tomographic scanning involves transporting critically ill or anesthetized patients to the scanner—which may not be practical—as well as significant cost and exposure to radiation.

The use of ultrasound in the diagnosis of pneumothorax has been widely reported in literature.1,2A pneumothorax-free ultrasound will show either “lung sliding” (a side-to-side movement of the parietal and visceral pleura) or a B-line artifact (hyperechoic reverberation artifacts arising from the pleural line because of small subpleural water-rich structures surrounded by air, previously known as “comet-tails”). A patient with either lung sliding or B-lines is pneumothorax-free (negative predictive value 100%), whereas absence of both indicates a high likelihood of pneumothorax (96.5% specificity).3 

The high specificity and sensitivity of ultrasound in identifying pneumothorax, combined with ultrasound’s portability, speed of use, and lack of radiation make it an attractive tool to rule out pneumothorax in the perioperative setting. However, in most of the previous studies documenting ultrasound’s validity, the examination has been done by experienced ultrasonographers3–5or by physicians who have undergone extensive training.6,7 

To achieve widespread use in an operating room or intensive care unit setting, the skill to detect pneumothorax on ultrasound must be easily acquired by new learners and retained with minimal effort. We tested the hypothesis that, with basic training in the identification of pneumothorax on ultrasound, anesthesia residents and faculty can rule out pneumothorax with a high degree of sensitivity and specificity when presented with an optimal ultrasound image of the chest. Additionally, we sought to determine the effect of prior ultrasound experience on their ability to detect pneumothorax using this modality.

After approval from the Institutional Review Board of the University of Iowa, Iowa City, Iowa, we obtained consent from 53 patients undergoing elective thoracic surgery that involved thoracoscopy or thoracotomy with intraoperative lung isolation for pulmonary wedge resection or lobectomy. Patients with pleural disease or pneumothorax on preoperative computerized tomographic scan were excluded from this study. Ultrasound video images were obtained using either the S12-4 or L12-5 surface probe (Philips Medical Systems, Andover, MA). The probe was oriented in a longitudinal axis on the anterior or lateral chest wall in approximately the fifth to eighth intercostal space, with a rib cross-section visible on either end of the video if possible (see Supplemental Digital Content 1, https://links.lww.com/ALN/A893). Depth, gain, compression, frequency, and focus were adjusted to acquire the best possible image with maximal visualization of the pleura and an ultrasound scanning depth 2–3 cm deeper than the pleura. Two video images were acquired per patient. The first video image (normal, i.e. , depicting absence of pneumothorax) was acquired immediately after induction of anesthesia. The second image (depicting pneumothorax) was acquired by the surgeons with an ultrasound probe in a sterile sheath, under guidance from the study team. This second image was obtained by placing the probe on the chest wall on the operative side toward the end of the surgical procedure, after visual confirmation of a collapsed lung. In all, 99 videos were compiled in a library for use during the quiz for anesthesiologists: 52 without pneumothorax and 47 with pneumothorax. Due to surgical time constraints, seven videos could not be obtained (six with pneumothorax and one without pneumothorax). The duration of each video was 3–5 s. Presence or absence of obesity (defined as body mass index >30) and chronic obstructive pulmonary disease was recorded for each patient. All the videos in the library were graded by one of the investigators (S.K.) for the presence of heartbeat or probe movement and for the subjective clarity (“quality”) of the ultrasound image. Videos were classified to be of poor quality if the ribs, pleural line, and subcutaneous tissue were not visualized well. Supplemental Digital Content 1–6 (https://links.lww.com/ALN/A893, https://links.lww.com/ALN/A894, https://links.lww.com/ALN/A895, https://links.lww.com/ALN/A896, https://links.lww.com/ALN/A897, https://links.lww.com/ALN/A898) provide examples of lung sliding, B-lines, pneumothorax, heartbeat, probe movement, and poor imaging, respectively. Supplemental Digital Content 7 and 8 (https://links.lww.com/ALN/A899, https://links.lww.com/ALN/A900) provide examples of images from patients with obesity and chronic obstructive pulmonary disease, respectively. In Supplemental Digital Content 1–8, subcutaneous tissue is marked with a yellow arrow, ribs in cross-section with white arrows, and the pleural line with a blue arrow.

Forty-eight residents and 53 faculty members from our anesthesia department were invited to participate. Physicians who agreed to enroll in this study then received instruction about the use of ultrasound to detect pneumothorax via  a 5-min online presentation (Supplemental Digital Content 9, https://links.lww.com/ALN/A901). This presentation explained the basic physics of ultrasound, the placement of the probe on the chest, and the basis of the various structures and artifacts visualized on chest ultrasound (ribs, pleural line, lung sliding, and B-lines). The presentation also included videos with lung sliding and B-lines, and a video with both these signs absent. An algorithm explaining the application of these ultrasound features was included in the tutorial (fig. 1). A still-frame example of B-lines is shown in figure 2.

After the tutorial, anesthesia residents and faculty were asked to take an online quiz with 20 ultrasound videos selected randomly for each participant from the aforementioned library of 99 videos. The quiz and randomization was set up on the University of Iowa’s ICON (Iowa Courses ONline, Desire2Learn, Inc., Kitchener, Ontario, Canada) Web site, which allows password-protected access to participants, with computerized randomization of quiz questions and tracking of individual performance. For each of the 20 ultrasound videos in the quiz, the physician had to decide whether a pneumothorax was present or not. Participants were not informed about the prevalence of pneumothorax in the collection of images in the video library.

The following demographics were recorded for anesthesia residents and faculty participating in the testing: years of anesthesia experience and description of any training in the use of ultrasound (for vascular access, echocardiography, or regional anesthesia) and frequency of ultrasound use (did not use ultrasound for any purpose [e.g. , vascular cannulation, regional anesthesia] in the last month, used ultrasound once or twice in the last month, or used ultrasound once or twice every week in the last month).

Previous studies on skill retention have tested trainees 6 months after initial training.8,9On the basis of these studies, we asked participants in our study to take a follow-up quiz 6 months later. During this second quiz, physicians were asked to rule out pneumothorax in another set of 20 ultrasound videos randomly selected from the video library. Participants were also asked whether they had used ultrasound for detection of pneumothorax in the intervening period between the quizzes.

The agreement between participants’ ability to rule out pneumothorax and the true state of the disease was evaluated by sensitivity and specificity. Sensitivity and specificity are tested for overall results (assuming all 1,580 answers are independent) and boxplots are presented to reflect the distribution of sensitivity and specificity on the user level (for 79 users). Using IBM SPSS Statistics 19 (Chicago, IL), univariate analysis with chi-square and Fisher exact test was performed to identify resident, patient, and ultrasound image factors that influenced the possibility of correctly ruling out pneumothorax, with a P  value of less than 0.05 considered significant. In this study, we explored the effects of anesthesia training status, prior ultrasound training, frequency of use of ultrasound, patient history of obesity or chronic obstructive pulmonary disease, presence of heartbeat or probe movement on the ultrasound video, and quality of the ultrasound video on the ability of anesthesia residents and faculty to correctly rule out pneumothorax. Power calculation was not done to estimate sample size. We intended to recruit all, or as many as possible, residents and faculty members of our anesthesia department.

Residents and faculty members who reported not having received any ultrasound training were grouped against those who reported training with ultrasound for central venous access, regional anesthesia, or echocardiography. Residents and faculty members who did not use ultrasound in the month before the quiz were defined as infrequent users of ultrasound, and compared with those who had used ultrasound at least once in the month before the quiz. For the second quiz, good retention was defined as a quiz score equal to or higher than that of the first quiz. Ultrasound training, anesthesia training status (residents vs.  faculty), and use of ultrasound for lung examination were tested as potential variables affecting retention.

To take into account the correlated nature of 20 scores within each user, generalized estimating equations was used in SAS (Cary, NC) by PROC GENMOD procedure by using repeated statement and unstructured correlation structure. Factors with a univariate P  value less than 0.2 on the chi-square test were treated as potential covariates in this model to identify significant covariates that independently affected the physicians’ ability to correctly rule out pneumothorax. It was assumed that two videos selected from the same patient were independent. A backward model selection was performed until all the covariates in the model had a P  value of less than 0.05. Interactions of significant main effects were also tested. SAS 9.3 XP_PRO platform was used for the generalized estimating equation model analysis.

Of 101 eligible anesthesiologists, 79 (78.2%) took part in the study. Pneumothorax was correctly ruled out in 732 out of 845 attempts (sensitivity 86.6%), and it was correctly identified in 629 out of 735 attempts (specificity 85.6%) (table 1). In the follow-up quiz at 6 months, 70 physicians participated. The mean score was 86.1% in the first quiz and 86.4% in the second quiz, with median scores of 90% (interquartile range, 80–95%) in both the first and the second quiz (fig. 3). Most of the participants were clustered in the 80–100% range for sensitivity and specificity in ruling out pneumothorax (fig. 4).

In the follow-up quiz at 6 months, 57% of participants had a score equal to or higher than their score in the first quiz. Nine participants (13%) had a score decline of more than 10% compared with their score in the first quiz, whereas 61 of 70 (87%) participants’ scores declined 10% or less compared with their first scores. The median difference in score from the first quiz to the second quiz was 0 (range, −13 to 10, interquartile range, −2 to 2). No factors were significant for retention of the skill to detect signs of pneumothorax on prerecorded ultrasound examination of the chest.

Physician, patient and ultrasound imaging variables characteristics are provided in table 2. On univariate analysis, infrequent use of ultrasound by the physician (P = 0.04), presence of probe movement (P = 0.02) or heartbeat (P = 0.001) on the ultrasound video and poor quality ultrasound imaging (P = 0.007) were significant factors in predicting the incorrect identification of ultrasound features of pneumothorax (table 3). These four covariates were tested in the generalized estimating equation model for their independent association with the ability to rule out pneumothorax. Presence of probe movement (P = 0.002) and heartbeat (P < 0.001) on the ultrasound video were independent covariates that significantly reduced the ability to correctly identify ultrasound features of pneumothorax (table 3). Presence of heartbeat on the ultrasound video led to more incorrect answers (sensitivity for ruling out pneumothorax 48.6%, specificity 77.8%) compared with videos with no heartbeat (sensitivity 88.4%; specificity 86.4%). Only videos with pneumothorax had probe movement in them. The presence of probe movement on the ultrasound video was associated with fewer correct answers (specificity 23.8%) compared with videos with no probe movement (specificity 86.5%).

Factors that did not independently affect the physicians’ performance on the quiz include prior training in or recent use of ultrasound, status of anesthesia training, patient history of obesity or chronic obstructive pulmonary disease, and poor ultrasound image quality.

In this study, we analyzed the impact of minimal training on the ability of medical providers to rule out pneumothorax on an optimal ultrasound image. After a 5-min online tutorial, anesthesia residents and faculty members were able to rule out pneumothorax on ultrasound with a high sensitivity and high specificity. They also demonstrated retention of this skill 6 months after training. Although the positive and negative predictive values (table 1) for our study might not be reliable because of the artificial 47% prevalence of pneumothorax in the video library, sensitivity and specificity still are largely independent of prevalence.

Numerous investigations have compared the utility of ultrasound to current methods such as chest x-ray and computerized tomographic scan in diagnosing pneumothorax. Sartori et al.  10reported 100% sensitivity in detection of pneumothorax after lung biopsy using ultrasound, whereas chest x-ray identified only 87.5% of these pneumothoraces. A recent review11of four prospective studies showed bedside thoracic ultrasound to be a more sensitive screening test for detection of pneumothorax in adult patients with blunt chest trauma when compared with supine chest radiography. Point-of-care lung ultrasound was also discussed in detail in recent recommendations.12 

Focused assessment with sonography for trauma is commonly used in the emergency setting. Emergency medicine and trauma authors have suggested extending the ultrasound evaluation to include the thoracic cavity in the search for pneumothorax, called the extended assessment with sonography for trauma. This extended assessment has been shown to be more sensitive than chest x-ray in detecting occult pneumothoraces after trauma.13Ultrasound also demonstrated utility in diagnosing pneumothoraces that were missed by chest radiographs, in an investigation in the intensive care setting.5 

Since the training of our residents and faculty members in the use of ultrasound for detection of pneumothorax, this modality has been used in multiple cases in our institution with immediate impact on patient care. These include the detection of pneumothorax during laparoscopic Nissen fundoplication, in a patient with polytrauma,14after placement of a subclavian central line, postoperatively in a cardiac surgical patient and after infraclavicular brachial plexus block. The degree of pneumothoraces in these settings varied from partial to total lung collapse. This experience indicates that not only can providers be easily trained in the use of ultrasound, but can also successfully apply their knowledge in a clinical setting.

There was a 15% chance of an incorrect diagnosis in this study. This could be potentially fatal when pneumothorax is suspected. The sensitivity of ultrasound to detect pneumothorax ranges from 86 to 100% in clinical studies.7,10For training, anesthesia residents and faculty members in our study watched a 5-min presentation with video examples of lung sliding, B-lines, and the absence of these signs. In comparison, Zhang et al.  7trained three emergency medicine physicians with 28 h of specialized training on ultrasound use before testing. The diagnostic sensitivity and specificity of ultrasound for diagnosis of pneumothorax was 86% and 97% respectively. Dulchavsky et al.  6investigated the ultrasound skills of surgical residents who were familiar with the use of ultrasound for abdominal trauma and who had attended a formal ultrasound course using video instruction and use of ultrasound on models, resulting in a 95% sensitivity in the diagnosis of pneumothorax. It is possible that more detailed training and further practice would have allowed infrequent ultrasound users in our study to acquire this skill better.

The high percentage of enrollment and the participants’ high accuracy rate on the quiz in our study demonstrates the value of an e-learning model with focused teaching of specific skills in short lectures. Participants accessed the 5-min presentation via  the University of Iowa’s Department of Anesthesia Intranet Web site. Similar teaching modules are available on the Internet on free Web sites.‖

Although our results are promising, an important step in transferring these skills to clinical practice is the ability to obtain good windows and images. The anesthesia residents and faculty in this study read the videos “off-line.” Actual probe placement on the patient’s chest and acquisition of ultrasound image are relevant skills that were not evaluated in this study. Also, multiple images were obtained per patient. Only those images that illustrated the presence or absence of pneumothorax best were included in the quiz. The aim of the current study was to assess the accuracy of pneumothorax detection when participants were given an optimal ultrasound image. Further investigation is necessary to assess the ability of trainees to acquire good images and distinguish image quality.

The pneumothoraces evaluated in this study involved completely or almost completely collapsed lungs. It is not possible to detect the magnitude of pneumothorax with ultrasound. At best, the presence of a “lung point” on ultrasound indicates that some part of the lung is not collapsed.15We did not use M-mode in our study design. M-mode signs are well-recognized methods for detecting lung sliding.16We chose to train and test one single method of imaging (two-dimensional) to keep the instruction simple for infrequent users of ultrasound.

In this study, heartbeat or probe movement on the prerecorded ultrasound video image influenced physician performance on the quiz. These factors can resemble lung sliding, leading to incorrect diagnosis. In the quiz, participants were shown one 5-s clip for each examination. In clinical practice, the limitations of probe movement and heartbeat mimicking lung sliding should be mitigated with a more thorough examination and with the ultrasound examiner’s knowledge of probe location and movement.

A majority of our physicians had undergone ultrasound training for central venous access, echocardiography, or regional anesthesia (table 4). It is conceivable that a 5-min training module would not suffice for training groups of physicians who are not well versed in ultrasound.

In conclusion, after viewing a 5-min online training video, physicians can reliably rule out pneumothorax on an optimal ultrasound image. They are also able to retain this skill for up to 6 months. Heartbeat and probe movement can be challenging features on ultrasound image when trying to identify pneumothorax.

The authors acknowledge assistance from Emine O. Bayman, Ph.D., Assistant Professor, and Michael M. Todd, M.D., Professor and Head, Department of Anesthesia, University of Iowa Hospitals and Clinics, Iowa City, Iowa.